Repository navigation
Added dynamic resampling - #225
Conversation
|
are there recommended values for the new settings? |
|
I would leave the defaults, but they are there if you need to tune them @alatteri |
|
@alatteri - To expand on what Ben has said. The default for the targetbuffer for a Decklink is currently set to 2700 samples (per channel). The drift code itself actually will try to stay within 600 samples of this target before it will even attempt to resample. You can tweak it but my recommendation is to stick with the default as this got us good results. The minresample is set to be 1Hz by default. The maxresample is set to be 30Hz by default. These are deltas from the existing sample rate for however you intend to sway the buffer. As the drift happens over a very long period of time, you generally should not have a need to resample more than this. As mentioned above, no resampling even occurs if you're inside of the target range (default 2100 - 3300 samples per channel in the buffer). Once you leave, the drift fixing class will calculate the distance you are away from this range and will resample accordingly. It scales the distance between (100 and 600 samples) and applies a min and max resample delta based on this. So at a distance of 100 samples it would apply a 1Hz (or whatever minresample is set to) delta to try and correct the drift. At a distance of 600 it will apply 30Hz (or whatever maxresample is set to). Currently I haven't set either the distance from the target buffer or the min/max of the scaling calculation to be parameterised, but the values work well enough during our testing. As said, the recommendation is to leave them as the defaults as our testing has been fairly successful so far, but hopefully the above nicely explains what the defaults are and the impact they have when you change them. |
|
@aw-sohonet |
|
@MartinPulec does this make sense to you ? |
f253344 to
12c6c76
Compare
|
I think it may be merged sooner or later. However, the merge request has 40 commits while altering also some of core audio processing internals, so we definitely need to review it. We may also want to slightly modify the code, eg. these:
Just for curiosity, why I am author of bbc9184? It is possible that I've done something wrong to commandeer it, but it seems to be so also in the original repo. |
|
Hi @MartinPulec, Thanks for getting back! Just to respond to a couple of your points:
I tried merging all of the commits into a single commit. I was attempting to exclude the change you made at the beginning of the branch and only merge down my commits, but that did not work and instead it added an additional commit (which included your initial commits on the branch). |
|
Hey everyone.... Any movement on this? It would be a great benefit to get this mainlined as we still get the buffer overflows with Decklink. |
|
Would it be possible rather to push as a merge request only the unmerged branch (eg. commit b53aa63)? I think that ideally the merge request should be a straight branch without branching. |
|
Hi Martin. Sorry I'm not entirely sure what you mean. The commit I can roll back this PR, to be at the commit you asked for by rewriting the history, but I would then need to recommit the changes I have made in order to resolve the conflicts (which I notice there are more since I last resolved the conflicts). Please let me know if I have interpreted what you are saying incorrectly! |
|
I think that it is a bit misunderstanding - I meant the commit b53aa63, not b39af61 (the merge commit), unfortunately the hashes are similar. In my opinion, the pull request doesn't need to be merged/mergeable. If you do the merge with master, after the merge of the pull request I'd need it once again, since has been merged with an older version. I've just tried to request a pull requeste #246 and it seems that only decklink.cpp is conflicting. Is it be possible to issue the pull request like that? |
b39af61 to
b53aa63
Compare
|
Hi Martin, I have forced the commit back to the hash that you asked for! It should now as I left it without the merged changes from |
exactly! thanks anyways, could you make one more change? Would it be possible to remove the commit bbc9184? I believe that this is just squashed commits 44ae96d..3a6ad84 (44ae96d is a common ancestor for both). Both references, which contain the same code, are then merged together. I've already tried the removal in #247 and it works flawlessly. Or is there a reason to have merged together the same set of changes once squashed and in the other branch unsquashed? If bbc9184 is held for the summary commit message, I'd see better options, like creating an extra commit with that commit message and eg. dropping a line to NEWS. |
Although the conversion shouldn't be set (see CESNET#215), still a valid value ('none' here) must be used for the check. Fixes CESNET#221
…and removed all unecessary log lines as part of the investigation into the code.
…ng issues to remove compiler warnings.
…w-latency. Made the summary output look more like other examples.
…r fill for the audio drifting class
… a log line to better reflect the error case it was covering.
* Added soxr 0.1.3 as an external dependency (for the resampling code) * Removed resampler quality parameter * The decklink drift fixing class has been changed to be disabled by default. The help for the decklink has also been updated to include details about the dynamic resampling process. * Stopped the dynamic resampler spamming the verbose log too much
|
@MartinPulec - I have done as you asked and removed the squash commit. It appears to have deleted every comment. |
I think it doesn't matter much. Anyways, sorry for the remarks but I don't think it is ideal yet. I think that the merge commit is still there (bbc9184) but also most commits go to the final merge in 2 paths, eg b53aa63 and 11efbce (and are perhaps also included in the squash). Would it be possible to do something like (or exactly so as) the tag merge-test2? You can look at it with either The first one is almost linear, while the second is complex - it may not be a problem if those were independent changes, but here the point is that most changes are there included in 2 (or even 3) paths. |
0ce6215 to
11efbce
Compare
|
Hi @MartinPulec - The git graph when I pushed it looked like what you had requested. - Hopefully it looks like you want it to be 😄! Let me know if there's anything else that needs to be done. |
|
Hi everyone.... just wondering if we can get any movement on this... would be wonderful to have this fix in master. Thanks! |
|
Merged, with some additional changes described below. Functional:
Non-functional:
I wasn't sure about DecklinkAudioSummary calls in Finally, I didn't test thoroughly the drift-fixer merge and subsequent changes if it still work since I don't have appropriate test setup. But if something breaks it shouldn't be hard to fix. Feel free to post eventual changes as a a pull requests. |
What is the option to enable it?
Is the latency automatically accounted for, are do we need to do a manual delay option to resync the video? |
It is documented –
You mean if UltraGrid adds extra latency for video? No, it doesn't. I don't think it would make some sense to do it (proactively) because audio and video is processed independently so the A/V sync is a bit indeterminate (DeckLink audio buffer is 2 or even 3 video frames long). |
|
@MartinPulec - Thanks for completing the merge conflict work and merging! It is much appreciated 🎉 |
|
OK...so if we use sox, is the latency a consistent 20ms? Which means to maintain A/V since we would add "--audio-delay 20" or do we just need to test and adjust, or am I misunderstanding the situation?
|
Didn't tried myself but there is an evaluation showing that. I guess rather |
|
From our testing the 20ms latency is a little overblown, but obviously this depends on your hardware. From our testing, when using threading, the latency is anywhere between 6ms and 10ms (although I'm doing this from memory). I don't know if Martin restored the Speex resampler with the threading work I did, but that gives a significant boost on higher channel counts as well. The reason we did not use Speex though is that variable sample rates causes the resampler to insert a small anomaly when it was destroyed. |
Added a dynamic resampling method that will control the amount of samples in the buffer for Decklink cards so that any drift occurring that causes overflows or underflows can be avoided.
It's worth noting that while this fix works well for Decklink cards, for 32bit Audio support there is a conversion between 32Bit integers and floats so that the Speex float resampler can be used (before a conversion back). If not all cards produce 32Bit integer audio, then this will cause any resampling that happens with these cards to break.
In longer term testing this has produced desirable results, which removes any popping, or glitches in the audio.
The threaded performance improvements make it so that the highest quality resampling can be used with minimal impact. Threaded vs unthreaded for quality 10 gave an increase in performance of roughly 250% (across 16 channels of data).
Fixed
Changed
Added