1541 Commits

Author SHA1 Message Date
dancho
16a15b94ca Add Configration for Frame Extraction for specific SeekParameters
Expose ExoPlayer seek parameters via FrameExtractor API

PiperOrigin-RevId: 696449874
2024-11-14 02:48:05 -08:00
dancho
301ef207f2 Add pixel comparison to FrameExtractorTest
Add PSNR comparison with the output of MedaiMetadataRetriever.

PiperOrigin-RevId: 696190585
2024-11-13 10:43:36 -08:00
dancho
175dca41df Read Bitmap in ExperimentalFrameExtractor
Add a MatrixTransformation GlEffect to flip between
OpenGL and Bitmap coordinates

PiperOrigin-RevId: 696029842
2024-11-13 01:27:02 -08:00
sheenachhabra
59106bba1c Start muxer watchdog timer on export start
Earlier watchdog timer was started only after all the tracks are
added. For the cases where export is stuck before adding tracks,
the export would hang forever and will not timeout automatically.

With the changes, watchdog timer is started as soon as export is
started/resumed. Now if the pipeline is stuck before writing any samples,
it will timeout and report error to the caller.

Existing test case: c35a9d62ba/libraries/transformer/src/test/java/androidx/media3/transformer/MediaItemExportTest.java (L996)

PiperOrigin-RevId: 695781654
2024-11-12 10:19:13 -08:00
dancho
40f187e4b4 Block calls to FrameExtractor.release
Block calls to release() method, and handle
calls on the player looper or not.

PiperOrigin-RevId: 695697602
2024-11-12 05:49:19 -08:00
dancho
ed288fca46 Experimental frame extraction based on ExoPlayer
A skeleton implementation that doesn't actually return decoded frames.
In the current state, we use ExoPlayer to seek to a position, and
ExoPlayer.setVideoEffects to record the presentation time selected.

Seeking and processing frames are synchronized via ListenableFuture
callbacks.

PiperOrigin-RevId: 695691183
2024-11-12 05:18:01 -08:00
Googler
4acd1b970c Make MediaSource.Factory a nullable parameter.
In some cases, callers will need to pass in both BitmapLoader and Codec.DecoderFactory without specifying a custom MediaSource.Factory. Omitting the annotation will result in NULL_FOR_NONNULL_TYPE compilation errors in Kotlin.

PiperOrigin-RevId: 695481606
2024-11-11 14:26:14 -08:00
dancho
31ae260db4 Support alternative sample MIME types in MuxerWrapper
Enable transformer to transmux into alternative sample MIME types.
For example, some Dolby Vision profiles have a backwards-compatible
AVC or HEVC layer. MV-HEVC is backwards compatible with HEVC.

This change enables Transformer to transmux into the backwards compatible
format to improve compatibility with legacy APIs such as
MediaMetadataRetriever.

PiperOrigin-RevId: 693667597
2024-11-06 03:58:02 -08:00
ibaker
286273c10e Enable lint in tests for modules that require no fixes
PiperOrigin-RevId: 693313908
2024-11-05 05:40:03 -08:00
claincly
b0c6106882 Fix trim optimization logic when edit lists exist
Also makes muxer shift the first video timestamp to zero, if it's not.

The trim position should respect the media timeline.

For example in a video that is 10s long (without edit list), if an edit list
adds 1_000ms to each video sample, and trimming 100ms, here's the expected:

- The video duration is 10.9s (`10s + 1s edit - 0.1s trim`)
- The first video frame time would be at 0.9s (`1s edit - 0.1s trim`)

PiperOrigin-RevId: 692187399
2024-11-01 08:36:08 -07:00
kimvde
96b923b610 Fix failing portrait export test
The logic in assumeFormatsSupported assumes that portrait videos are
always rotated before encoding but that's not the case when portrait
encoding is enabled.

PiperOrigin-RevId: 691042881
2024-10-29 09:10:59 -07:00
kimvde
14094b5094 Fix offset passed to VideoFrameReleaseControl
In some cases, the streamOffsetUs was passed to
VideoFrameReleaseControl.getFrameReleaseAction() but it should be the
streamStartPositionUs.

PiperOrigin-RevId: 691040172
2024-10-29 09:04:35 -07:00
shahddaghash
7a8f05f736 Remove deprecated Transformer methods/variables
Removed deprecated `Transformer.PROGRESS_STATE_NO_TRANSFORMATION`, `Transformer.setListener`, and `Transformer.startTransformation` from Transformer.

PiperOrigin-RevId: 690971992
2024-10-29 05:02:31 -07:00
shahddaghash
51055d7e79 Remove deprecated TransformationException class.
Class was deprecated in Media3 1.1.0. Use `ExportException` class instead.

PiperOrigin-RevId: 690955853
2024-10-29 03:59:15 -07:00
shahddaghash
dd6e88889a Remove deprecated TransformationResult class.
Class was deprecated in Media3 1.1.0. Use `ExportResult` class instead.

PiperOrigin-RevId: 690694730
2024-10-28 11:51:02 -07:00
shahddaghash
772bd20f7d Remove deprecated Transformer.Listener methods
Methods were deprecated between Media3 1.0.0 and Media3 1.1.0.

PiperOrigin-RevId: 690656410
2024-10-28 10:09:20 -07:00
dancho
4601a42aee Use PSNR for comparing equality in one HDR10 test
Also, fix incorrectly stretched golden file that was not detected with previous, less sensitive, average pixel error comparison.

PiperOrigin-RevId: 690643520
2024-10-28 09:38:34 -07:00
dancho
fe14525a97 Use ExoPlayer in DecodeOneFrameUtil
The old code that uses MediaCodec directly has a race condition
that causes the decoder to incorrectly crop the decoded picture.

PiperOrigin-RevId: 690620868
2024-10-28 08:32:23 -07:00
shahddaghash
9e088ac2b8 Remove 7 deprecated Transformer.Builder methods
PiperOrigin-RevId: 690617093
2024-10-28 08:19:41 -07:00
dancho
5bb9e1a932 UltraHDR Ovelays: remove texture unit clash
Do not use the same texture unit for two different textures.
This was causing tests to fail on multiple devices.

PiperOrigin-RevId: 690610094
2024-10-28 07:59:09 -07:00
dancho
d2fb779929 Ignore flaky preview tests
Tests are flaky because shader program creation is slow

PiperOrigin-RevId: 689420905
2024-10-24 10:04:57 -07:00
dancho
94e37ca7d2 Fix flaky test
* Use asset without audio for independent video time progress
* Use longer item duration to avoid ExoPlayer STATE_READY workaround
  (see https://github.com/google/ExoPlayer/issues/1874)

PiperOrigin-RevId: 689409427
2024-10-24 09:29:35 -07:00
kimvde
be8c58d51e Use Format object in VideoFrameProcessor
This is to use an existing media3 object rather than creating a new one.

PiperOrigin-RevId: 688481323
2024-10-22 03:46:03 -07:00
shahddaghash
320cbc09f4 Remove deprecated AudioMixer.create() method
Use `DefaultAudioMixer.Factory().create()` instead.

PiperOrigin-RevId: 688167322
2024-10-21 09:39:34 -07:00
ibaker
b04b37074b Add video language to DefaultTrackSelector
PiperOrigin-RevId: 688155680
2024-10-21 09:02:19 -07:00
claincly
73790cf2a4 Don't optimize for trim if the media is not clipped
PiperOrigin-RevId: 688120868
2024-10-21 06:58:59 -07:00
claincly
2a49ffcb23 Make OverlaySettings dynamic
PiperOrigin-RevId: 687269731
2024-10-18 05:43:33 -07:00
shahddaghash
2e61c93dba Remove deprecated DefaultEncoderFactory constructors.
Use `DefaultEncoderFactory.Builder` instead.

PiperOrigin-RevId: 686821388
2024-10-17 01:53:23 -07:00
kimvde
31ef7ff088 Clarify Javadoc of EditedMediaItem.Builder.setDurationUs
PiperOrigin-RevId: 686521901
2024-10-16 08:48:10 -07:00
kimvde
9adb3aaf41 Transformer: add an entry point to disable automatic rotation
PiperOrigin-RevId: 686067527
2024-10-15 05:40:41 -07:00
sheenachhabra
407bd49ed5 Skip audio encoding bitrate setting test on API <= 23
The encoder output format on API 23 does not seem to contain bitrate,
hence the test fails.

PiperOrigin-RevId: 686047480
2024-10-15 04:15:38 -07:00
Googler
0100f1d902 Boxes: Add edit list box.
PiperOrigin-RevId: 685974308
2024-10-14 23:26:08 -07:00
claincly
17c0ff8ba8 Log warnings when Transformer sees unsupported track type
PiperOrigin-RevId: 685649866
2024-10-14 03:54:52 -07:00
sheenachhabra
a0ccd46653 Skip AACObjectHE encoding on faulty devices
These devices claim to have the AACObjectHE profile but the profile never gets applied.

PiperOrigin-RevId: 684786157
2024-10-11 04:18:52 -07:00
ibaker
2c46cea088 Use RoundingMode.DOWN in Util.scaleLargeTimestamp and friends
The implementation of these methods was updated from direct java integer
arithmetic in 885ddb167e.
In this change, `RoundingMode.FLOOR` was used to try and maintain
compatibility with java integer division. This was incorrect, because
java integer division uses `DOWN` (i.e. towards zero), rather than
`FLOOR` (i.e. towards negative infinity) semantics.

This change fixes the compatibility.

The dump file changes in this CL relate to tests that exercise edit
list behaviour. This involves manipulating negative timestamps, which
explains why they are impacted by this change.

PiperOrigin-RevId: 684013175
2024-10-09 06:24:26 -07:00
kimvde
52f08d46c2 Add motion photo support to Transformer
PiperOrigin-RevId: 683540867
2024-10-08 02:57:52 -07:00
ivanbuper
af922fbcb0 Fix truncation error accumulation on Sonic's time stretching algorithm
This CL also fixes EOS handling to account for not-yet-copied samples in
`remainingInputToCopyFrameCount`, which would throw off the final output
sample count calculation.

For testing, we allow a tolerance of 0.000017% drift between expected
and actual number of output samples. The value was obtained from running
100 iterations of `timeStretching_returnsExpectedNumberOfSamples()` and
calculating the average delta percentage between expected and actual
number of output samples. Roughly, this means a tolerance of 40 samples
on a 90 min mono stream @48KHz.

PiperOrigin-RevId: 683133461
2024-10-07 04:59:54 -07:00
dancho
7b08bedf2c Do not force EOS when decoder has produced all frames
The workaround in ExternalTextureManager.forceSignalEndOfStream
was being applied even when the decoder did output all frames, and only
the GL pipeline was slow.

This change ensures that workaround is not applied when the decoder
has already produced all output frames.

PiperOrigin-RevId: 680471587
2024-09-30 01:50:10 -07:00
claincly
e0e9f5b057 Increase the maxImage on ImageReader, hoping less flaky tests
PiperOrigin-RevId: 679528425
2024-09-27 04:03:15 -07:00
dancho
b9aed0a937 Have AndroidTestUtil.canEncode mirror VideoSampleExporter
Some devices can encode portrait 720x1080 but not landscape
1080x720. But VideoSampleExporter always prefers encoding
landscape. Have `assumeFormatsSupported` mirror sample exporter
logic more closely

PiperOrigin-RevId: 679495210
2024-09-27 02:01:05 -07:00
samrobinson
45c400c7b5 Clarify compositionPlayer audio test naming.
PiperOrigin-RevId: 679142528
2024-09-26 07:37:52 -07:00
sheenachhabra
b6192f7a39 Add AudioEncoderSettings to Transformer
It allows clients to specify the audio encoding profile and bitrate.
This is similar to VideoEncoderSettings.

PiperOrigin-RevId: 679131963
2024-09-26 06:59:53 -07:00
claincly
50c879ee21 Remove stale comment
PiperOrigin-RevId: 679082395
2024-09-26 03:55:24 -07:00
Googler
020ce7765c Reduce rounding error and stts table entries.
To avoid rounding errors, set the `Rounding mode` of the `uvFromVu` and `vuFromUs` results to `HALF_UP`. This `Rounding mode` rounds numbers towards the "nearest neighbor" unless both neighbors are equidistant, in which case round up.

PiperOrigin-RevId: 679003943
2024-09-25 23:19:20 -07:00
claincly
0ea63e3fa6 Disable video only on the sequence that has items that disable video
This means we need a custom track selector for each SequencePlayer.

PiperOrigin-RevId: 678779216
2024-09-25 11:28:26 -07:00
kimvde
3ec9c99644 Document that the MediaItem's image duration should be set for images
PiperOrigin-RevId: 678702334
2024-09-25 08:04:31 -07:00
kimvde
2dc32360d6 Remove usages of deprecated DefaultDecoderFactory constructor
The constructor was deprecated but usages were not migrated.

PiperOrigin-RevId: 678683678
2024-09-25 07:01:14 -07:00
samrobinson
fa04386863 Enable multi sequence audio preview test.
PiperOrigin-RevId: 678667829
2024-09-25 06:03:23 -07:00
samrobinson
5ab72a3938 Use constants in dump file names for playback test.
PiperOrigin-RevId: 678664730
2024-09-25 05:51:06 -07:00
kimvde
eaec7a4a61 Set MediaItem's image duration for image export
This field will become mandatory

PiperOrigin-RevId: 678656879
2024-09-25 05:23:27 -07:00