The video asset loader renders decoder output to a surface texture, and if the
video sample pipeline is in the process of updating the surface texture image
at the moment when the asset loader video decoder is released this seems to
cause `MediaCodec.release` to get stuck.
Swap the release order so that we stop updating the texture before trying to
release the codec.
PiperOrigin-RevId: 523401619
Log at debug level immediately when MediaCodec throws. This logging will be
output closer to the time when the error actually happened so should make it
easier to identify the order of components failing.
Downgrade logging of errors after export ends to warning level, as output may
still be fine if there was a problem after exporting completed (though it's
still worth logging a warning as the device may not be in a good state).
PiperOrigin-RevId: 523370457
Before this CL, SurfaceTexture.onFrameAvailable was used to tell whether a frame
was available in the VideoFrameProcessor's output texture. This was incorrect, as
it would rely on having the texture be written to before the
SurfaceTexture.onFrameAvailableListener is invoked, leading to null-pointer-
exceptions on timeouts.
Instead of using DefaultVideoFrameProcessor different interfaces to set that we
want to output to a texture, and get that output texture, use one interface that
sets a listener, and renders to a texture iff that listener is set. As this
listener is executed on the GL thread, this also allows us to no longer need to
expand visibility for the GL task executor and tasks.
PiperOrigin-RevId: 522362101
Handling of the stream offset and start position was unnecessarily
complex and even incorrect. It was going to be an issue for
concatenation of video and image input.
The stream offset is the offset added before decoding/encoding to
make sure it doesn’t fail in case of negative timestamps (which do
rarely occur).
The start position is equal to the stream offset, plus the clipping
start time if the media is clipped.
Before this change:
- Samples were offset by the start position before decoding, and this
offset was removed before muxing.
- The startPosition of the first MediaItem in a sequence was used for
all the MediaItems in this sequence (which is incorrect).
- The stream offset was removed before applying the GL effects and
added back before encoding so that it was not visible to the OpenGL
processing.
After this change:
- The start position is subtracted in the AssetLoader, so that the
downstream components don’t have to deal with the stream offsets and
start positions.
- Decoded samples with negative timestamps are not passed to the
SamplePipelines. The MediaMuxer doesn’t handle negative timestamps
well. If a stream is 10 secondes long and starts at timestamp -2
seconds, the output will only contain the samples corresponding to the
first 8 (10 - 2) seconds. It won’t contain the last 2 seconds of the
stream. It seems acceptable to remove the first 2 seconds instead.
PiperOrigin-RevId: 520916464
Releasing the player once a sequence has ended seems to make our
emulator tests flaky. Comment out until we find the cause. The player
will still be released from TransformerInternal, when the export ends.
PiperOrigin-RevId: 520886181
*** Original commit ***
Rollback of fc539da061
*** Original commit ***
Release SequenceAssetLoader when ended
Before, the SequenceAssetLoader was released at the end of the export.
Release resources earlier if possible.
***
***
PiperOrigin-RevId: 520036433
*** Original commit ***
Release SequenceAssetLoader when ended
Before, the SequenceAssetLoader was released at the end of the export.
Release resources earlier if possible.
***
PiperOrigin-RevId: 519993003
Previously, we always used ImageReader to read from the output of DefaultVideoFrameProcessor, for pixel tests. This has a limitation of not being
able to read HDR contents, so that we couldn't support HDR pixel tests.
Reading from a texture allows us to use glReadPixels to read from
DefaultVideoFrameProcessor, and build upon this to implement HDR pixel tests. We do
still want tests for surface output though, because real use-cases only will output
to Surfaces.
Also, add some tests for outputting to textures, since this test infrastructure is
a bit complex.
PiperOrigin-RevId: 519786535
The failure reason (no network) will be logged in the test output, but the test will not be marked as failing, rather skipped.
PiperOrigin-RevId: 518262203
This was trimmed using the Transformer Demo, from ~4 seconds to 300ms
Otherwise, this test file will be quite large, and folks who clone the repo will
have to clone this content as well. Trimming the file also allows the test to
take less time, while still testing the core purpose of the file, to confirm that 4k extraction/decode/GL/encode/muxing works.
PiperOrigin-RevId: 517408463
This test file used to be remote, which led to various errors in loading the file or timing out during the muxer or overall transformer.
export4k60 test timing on Pixel 7:
|Condition |elapsedTimeMs|AS Test "Duration" (s)|
|--------------------------|-------------|----------------------|
|local file (this CL) |4253 |14 |
|Remote file, googleguest |5510 |17 |
|Remote file, 4G |11423 |25 |
PiperOrigin-RevId: 516815462
- To support looping EditedMediaItemSequences, we need a way to tell the
AssetLoader that a sample couldn't be consumed and that it should retry
later. This is necessary in case we don't know yet whether the looping
sequence should load more samples because the other sequences haven't
made sufficient progress yet.
- The decision on whether to consume a sample is based on its timestamp
so it needs to be available.
PiperOrigin-RevId: 516546026
This test is not run on emulator, triage or postsubmit, and does not
provide value to the library in it's current state. It could be
deleted, however it still may have use for analysis of exported files
on MH.
PiperOrigin-RevId: 516491160
Split test/ TransformerEndToEndTest into SingleMediaItemEndToEndTest and
SingleSequenceEndToEndTest to reduce the file size and split the tests
by category.
PiperOrigin-RevId: 515039502
Callbacks onTrackCount and onTrackAdded can be called simultaneously
from different threads.
Before this fix, it was possible for the MuxerWrapper and
FallbackListener track count to never be set, or to be set
with incorrect values.
PiperOrigin-RevId: 514779719
- This is to make sure we know about all the tracks before initializing
the SamplePipelines. This allows to set the muxer and the fallback
listener track count before the SamplePipelines are built.
- As a result, the test files had to be updated because the order in
which the tracks are written has changed.
- The ImageAssetLoader also had to be updated to call onOutputFormat
repeatedly until it returns a non-null SampleConsumer.
- Also fix the trackCount sent to the muxer and fallback listener. The
correct track count can be computed now that we know about all the
tracks before building the SamplePipelines.
PiperOrigin-RevId: 514426123
All audio tracks should either all be transcoded or all be transmuxed.
Same for video tracks.
To achieve this, simplify the behaviour of transmuxAudio/Video.
PiperOrigin-RevId: 513809287
