This means we now require 2+ input frames per input, and compare the primary
stream timestamp with secondary stream timestamps in order to select the
correct output timestamp. We also must release frames and back-pressure as
soon as possible to avoid blocking upstream VFPs.
Also, improve signalling of VFP onReadyToAcceptInputFrame
PiperOrigin-RevId: 553448965
This allows for custom implementations of this interface, like
a TestVideoCompositor or partner-implemented implementation
PiperOrigin-RevId: 552541631
After this change, every queued bitmap is treated as an individual input stream
(like a new MediaItems).
This change merges the FrameDropTest and FrameDropPixelTest into one (while maintaining all the test cases)
- This is accomplished by generating bitmaps with timestamps on it in FrameDropTest and compare them with goldens (one may call this a pixel test, please lmk if you want this to be renamed)
- The most part of the change comes from DefaultVideoFrameProcessorVideoFrameRenderingTest. The overall working is
- We bypass the input manager
- The TestFrameGenerator generates frames based on timestamps. In this case, we generate frames with timestamps on it
- The generated frame is sent to texture output and in turn saved to bitmaps
- We then compare the generated bitmap with the goldens
PiperOrigin-RevId: 551795770
For each event, the timestamp and presentation time is logged. The trace can
then be dumped to a tsv file and easily imported in a spreadsheet.
PiperOrigin-RevId: 550839156
signalEndOfInputStream is needed for when streams have different amounts of
frames, so that if the primary stream finishes after a secondary stream, it
can end without waiting indefinitely for the secondary stream's matching
timestamps.
onEnded mirrors this API on the output side, which will be necessary to
know when to call signalEndOfInput on downstream components (ex. on downstream)
VideoFrameProcessors
PiperOrigin-RevId: 549969933
The VideoFrameProcessingTaskExecutor already handles wrapping
GlUtil.GlException errors into VideoFrameProcessingExceptions.
Therefore, stop wrapping this ourselves, so that errors are
attributed to the correct line number with less indirection.
PiperOrigin-RevId: 549959258
Alters the OverlayShaderProgram implementation to support rotations, however we need to apply the transformations separately in order for them to work as expected so the matrix is removed from the interface in favour of explicit methods.
Adds a rotation test to ensure this ability doesn't regress
PiperOrigin-RevId: 549890847
For now, just identical timestamps. A follow-up CL will allow for different
framerates. Prior tests already send identical timestamp frames, so no new
tests were added
PiperOrigin-RevId: 548676759
Before this CL, the text with a scale span would wrap text, with because the scale wasn't taken into account when measuring the width of the overlay.
PiperOrigin-RevId: 548123626
* Allow more than one input bitmap at a time.
* Allow Compositor to take in and set up an Executor. Otherwise,
Compositor resources may be created on one thread and accessed on another.
* Add a Compositor TestRunner to reuse test code more.
* Update VideoFrameProcessingTaskExecutor to use a new onError listener, so
that it's more reusable in non-DVFP contexts, like for Compositor.
PiperOrigin-RevId: 547206053
Frame buffers are focused, not current. EGL Contexts may be current but this
is a prerequisite for focusing a framebuffer, and this alone isn't
sufficient to make a bitmap.
Clear this up as it can make some complex tests harder to understand, if the
methods needed don't match up in naming.
PiperOrigin-RevId: 546305118
BaseGlShaderProgram and SingleFrameGlShaderProgram are technically color space agnostic. useHdr is passed to the TexturePool, which only uses the value to choose between high/regular precision. useHdr is therefore a bit misleading and should be generalized to "useHighPrecisionColorComponents."
PiperOrigin-RevId: 545726629
This allows us to simplify and reduce thread initialization, hops and the
synchronization cost associated with them, especially when more and more VFPs
are used.
PiperOrigin-RevId: 545622169
Providing the sync token in the api allows the client to decide which waiting method they would like to use depending on the use case, allowing them to optimise if possible.
PiperOrigin-RevId: 543997311
MP4 edit lists sometimes ask to start playback between two samples.
If this happens, we currently change the timestamp of the first
sample to zero to trim it (e.g. to display the first frame for a
slightly shorter period of time). However, we can't do this to audio
samples are they have an inherent duration and trimming them this
way is not possible.
#minor-release
PiperOrigin-RevId: 543420218
By passing this class where it's needed, implementations don't need to store it
in a field (reducing boilerplate) and it's clearer that it can't be unset when
needed.
PiperOrigin-RevId: 542823522
To more accurately describe what they do, especially as Compositor will
starts to use more contexts or threads, and it's important to know what
needs to be reset/recreated/focused before what methods.
PiperOrigin-RevId: 541010135
Previously, TextureMangers have a method to signal ending of a current input
stream, and a method to end the **entire input**. The responsibility of both
methods are not easy to document, understand and read.
With the new design,
- Only `TextureManager.signalEndOfCurrentInputStream()` is kept
- It's called for every MediaItem in the sequence, include the final one
- FinalWrapper now takes explicit signal that frame processing is ending,
rather than relying on the return value of `onCurrentInputStreamProcessed()`
- On DVFP receiving EOS from the pipeline, it signals FinalWrapper the stream
is ending, **before** signaling the input switcher, so that FinalWrapper is
able to end the stream when the onCurrentInputStreamEnded signal eventually
reaches FinalWrapper
PiperOrigin-RevId: 540856680
This helps debug bitmap tests. Without this CL, I typically
need to go into the file to find the test name and absolute
path. With this CL, I can just adb pull the printed file path,
which is much simpler.
PiperOrigin-RevId: 540539042
The existing NullableType has been deprecated 5 years ago and causes
crashes in Kotlin apps because Kotlin doesn't recognize this annotation
as a nullable type annotation.
While we can't align on a single @Nullable annotation yet, we can at
least replace this one by JSR305's @Nonnull(MAYBE) as it fulfils all
requirements, including full Kotlin compatiblity. To avoid the
cumbersome name, we can redefine it as our own @NullableType
annotation. (We can't use @Nullable to avoid name clashes with the main
@Nullable annotation from AndroidX)
Issue: google/ExoPlayer#6792
PiperOrigin-RevId: 540497469
Partially roll forward of 5c29abbbf4, and adds some extra logic
Changes to the original CL
The original logic (a66f08ba97) fails in the following case:
> This is only seem on emulators.
- EOS is sent to ExtTexMgr
- The timer starts
- One frame arrives on SurfaceTexture, reset the timer
- The frame is sent for processing, now `availablFrames == 0`
- One frame arrives on Surface, reset the timer
- The frame is kept on SurfaceTexture for the downstream shader doesn't have
capacity, `availablFrames == 1`
- Timer times out as the downstream processor doesn't report being able to take
another frame.
- Although there's a frame available on the SurfaceTexture
This is solved by having the force EOS logic clear all the frames that the
SurfaceTexture holds. This also ensures the first frame dequeued from the next
stream isn't from the previous stream.
PiperOrigin-RevId: 540023359
This methods currently registers and queues the bitmaps Will need to have a method that just queues bitmaps in future CLs.
#minor-release
PiperOrigin-RevId: 539605628
