1771 Commits

Author SHA1 Message Date
shahddaghash
52db3a240d Clear cached encoders in ShadowMediaCodecConfig#after()
Instead of calling it in the tests setup, the change modifies ShadowMediaCodecConfig#after() method to call EncoderUtil#clearCachedEncoders().

PiperOrigin-RevId: 748625111
2025-04-17 03:51:13 -07:00
dancho
9fca713045 Ignore flaky test
PiperOrigin-RevId: 747793474
2025-04-15 03:06:51 -07:00
dancho
f261fe187a Assert frame counts with tolerance.
MediaCodec decoders sometimes output frames in
the wrong order. Make our asserts more permissive to
reduce noise in tests.

PiperOrigin-RevId: 747414696
2025-04-14 07:48:30 -07:00
shahddaghash
07be60ed93 Add encoder support to ShadowMediaCodecConfig
This is a step towards unifying ShadowMediaCodecConfig structure to accommodate having both ExoPlayer and Transcoding codecs configuration.

This includes:
* Adding ability to configure encoders by calling `addEncoders(CodecInfo...)`
* A new factory method that takes specific encoders and decoders CodecInfos
* A new method `addCodecs(boolean, CodecConfig, CodecInfo...) that configures codecs with specified behavior by passing a `CodecConfig`.

This CL also includes migrating Transformer tests to ShadowMediaCodecConfig.

PiperOrigin-RevId: 747390451
2025-04-14 06:30:04 -07:00
claincly
4bfa154acd Allow replays with CompositionPlayer
PiperOrigin-RevId: 745536073
2025-04-09 04:54:48 -07:00
claincly
e11a8a1b19 Fix playback not transition into next mediaItem
This is because the replay cache needs to clear the cache after one mediaItem is fully played.

That is, if the last two frames are cached, we need to wait until they are both rendered before
receiving inputs from the next input stream because the texture size might change. And when the
texture size changes, we teardown all previous used textures, and this causes a state confusion
in the shader program in that, the cache thinks one texture id is in-use (because it's not released)
but the baseGlShaderProgram texturePool thinks it's already free (as a result of size change)

Also fixes an issue that, if replaying a frame after EOS is signalled, the EOS
signal is lost because we flush the pipeline.

PiperOrigin-RevId: 745191032
2025-04-08 10:08:34 -07:00
sheenachhabra
72bb474604 Add experimental prefix to setForce{Audio,Video}Track APIs
These APIs are likely to change/removed in near future.

PiperOrigin-RevId: 743899703
2025-04-04 04:57:56 -07:00
claincly
9785c1fd92 Allow setting VideoFrameMetadataListener on CompositionPlayer
PiperOrigin-RevId: 743607386
2025-04-03 10:12:32 -07:00
sheenachhabra
1f3a5e7d1c Do not allow video transmuxing if there are video gaps
PiperOrigin-RevId: 743555458
2025-04-03 07:32:39 -07:00
kimvde
ee611a1ab8 Access finalAudioSink from playback thread
The finalAudioSink was accessed from the main thread but it should only
be accessed from the playback thread.

PiperOrigin-RevId: 743548450
2025-04-03 07:09:01 -07:00
kimvde
18e9a3fe36 Rename VideoSink.onStarted/Stopped
start/stopRendering better reflects what the method does.

PiperOrigin-RevId: 743493973
2025-04-03 03:42:57 -07:00
dancho
769aca2e32 Internal-only change
PiperOrigin-RevId: 743468059
2025-04-03 02:05:44 -07:00
sheenachhabra
bd14b753ee Add forceVideoTrack flag on EditedMediaItemSequence
This is similar to forceAudioTrack.

PiperOrigin-RevId: 743235372
2025-04-02 12:24:58 -07:00
ibaker
a9c0349214 Standardize percentage calculations
This is a follow-up to 72f5df582a

PiperOrigin-RevId: 743173295
2025-04-02 09:46:15 -07:00
kimvde
c8a34ec846 Handle RawAssetLoader feeding data while Transformer is being ended
PiperOrigin-RevId: 743017931
2025-04-02 00:58:28 -07:00
sheenachhabra
cf3faf9cff Validate gap at start when building sequence
PiperOrigin-RevId: 742710209
2025-04-01 08:31:37 -07:00
kimvde
812e078310 Make sure ExoPlayerAssetLoader's progress is not more than 100
PiperOrigin-RevId: 742710076
2025-04-01 08:29:44 -07:00
sheenachhabra
209ecce6b3 Change getVideoTrackOutput() to getTrackOutput()
The updated method can be used for audio as well.

PiperOrigin-RevId: 742673561
2025-04-01 06:36:03 -07:00
sheenachhabra
2141d9ef9c Make forceAudioTrack flag mandatory when gap is at start
Earlier when gap is at the start of a sequence
it was automatically filled with silent audio.
Now setting forceAudioTrack flag is mandatory to
indicate that the gap at the start of a sequence
must be filled with silent audio.

PiperOrigin-RevId: 742625236
2025-04-01 03:40:41 -07:00
sheenachhabra
8508e12001 Rename TransformerVideoGapsTest to TransformerGapsTest
This will allow keeping both audio and video gap test
in the same class.

PiperOrigin-RevId: 742291252
2025-03-31 09:27:57 -07:00
sheenachhabra
25c1760b17 Add VIDEO_APV mimetype to FrameworkMuxer supported types
Came up in github issue https://github.com/androidx/media/issues/2275.

PiperOrigin-RevId: 742233737
2025-03-31 05:44:06 -07:00
claincly
9254efd8da Add pixel test for replaying
I could've add another test that seeks into the media before replying, but I
don't think fundamentally it's different from the one added.

I wish I could add one that replays while playing, but it'd be hard to match
the frames perfectly.

I'll add more timestamp based tests

PiperOrigin-RevId: 742229436
2025-03-31 05:22:31 -07:00
kimvde
427daef350 Remove VideoSink.setWakeUpListener
This is the last Renderer reference in VideoSink

PiperOrigin-RevId: 742189332
2025-03-31 02:26:56 -07:00
sheenachhabra
c858abda5d Remove usage of Composition.Builder#experimentalSetForceAudioTrack
This CL removes all usages within media3.

PiperOrigin-RevId: 741469343
2025-03-28 03:51:27 -07:00
sheenachhabra
87e0d7b95a Add setForceAudioTrack method on EditedMediaItemSequence
PiperOrigin-RevId: 740808813
2025-03-26 10:04:14 -07:00
kimvde
12b4c7d780 Remove unused param from VideoSink
PiperOrigin-RevId: 740739740
2025-03-26 06:15:03 -07:00
kimvde
6cd15fb4b0 Remove PlaybackVideoGraphWrapper param from its listener methods
It's unclear why the listeners would need access to the
PlaybackVideoGraphWrapper as there should always be only one. Also,
it's not common practice to pass the parent object to the listener in
media3 (see Player.Listener for example).

PiperOrigin-RevId: 740686007
2025-03-26 02:32:51 -07:00
kimvde
0ad7ceadb3 Remove references to Renderers from VideoSink
Remaining references will be removed in follow-up changes

PiperOrigin-RevId: 740362812
2025-03-25 08:43:44 -07:00
kimvde
2642d895bd Remove VideoSink.onRendererEnabled
This is part of the effort to make VideoSink independent from renderers.

PiperOrigin-RevId: 740344126
2025-03-25 07:44:50 -07:00
Googler
6034a3c3d6 Tone map in CompositionPlayer if surface can't display HDR
PiperOrigin-RevId: 740290183
2025-03-25 04:17:50 -07:00
shahddaghash
5f7066a9d9 Resolve race condition and NPE for EditingMetricsCollectorTest
Previously, `exportSuccess_populatesEditingEndedEvent` and `exportError_populatesEditingEndedEvent` used an AtomicReference to capture the EditingEndedEvent. However, this led to a race condition where `EditingEndedEvent` was accessed before it was set in the `onMetricsReported` callback, resulting in a NullPointerException. This change replaces the AtomicReference with a SettableFuture to ensure the test waits for the event to be reported before asserting its value.

PiperOrigin-RevId: 739146511
2025-03-21 05:36:34 -07:00
shahddaghash
3f5019b908 Prevent metricsReporter.reportMetrics() from throwing
In general, metrics collection should not lead to Transformer thorwing and crashing the app.

PiperOrigin-RevId: 739099456
2025-03-21 02:02:44 -07:00
Googler
6280ab24c2 Scaffold CodecDbLite
CodecDbLite is a repository of chipset-specific codec recommendations.
Initially, these recommendations will focus on video encoders,
specifically choosing the right encoder to use on a given chipset
and the B-frame configuration to use with that encoder.

PiperOrigin-RevId: 738895394
2025-03-20 12:18:42 -07:00
claincly
cf0d9d207f Change wrong frame rate in test files
PiperOrigin-RevId: 738509242
2025-03-19 13:03:47 -07:00
claincly
9bc088bf2b Remove VisibleForTesting on setVideoGraphFactory
This enables apps to use MultipleInput, or customized VideoGraphFactorys

PiperOrigin-RevId: 738413586
2025-03-19 08:45:26 -07:00
claincly
8e56810b57 Use VideoFrameMetadataListener for replay notification
PiperOrigin-RevId: 738366305
2025-03-19 06:09:58 -07:00
kimvde
14c06eaf8e Remove implementation of DefaultVideoSink.onRendererEnabled
The goal is to get rid of VideoSink.onRendererEnabled so that this
interface becomes renderer-agnostic. Indeed, for multi video sequences,
the DefaultVideoSink won't receive input from a single renderer anymore.

This is a no-op refactoring

PiperOrigin-RevId: 738296515
2025-03-19 01:23:45 -07:00
kimvde
79f29680fc Ignore multi image sequence test
This test is failing in <unknown commit> when the following happens in
PlaybackVideoGraphWrapper:
- Both InputVideoSinks call onInputStreamChanged with startPosition
1000000000000
- Before the first VideoGraph output frame is available, both
InputVideoSinks call onInputStreamChanged with startPosition
1000000200000. This incorrectly erases startPosition
1000000000000 from TimedValueQueue streamStartPositionsUs. Indeed, when
1000000200000 is added for the second time, the whole queue is cleared
because this new value is not strictly greater than the previous one.

This start position logic should only be used for single-sequence
rendering. Disable the test until this is fixed.

PiperOrigin-RevId: 737982504
2025-03-18 06:35:06 -07:00
tonihei
0e169ab1be Move decode-only and no surface logic inside VideoFrameReleaseControl
This brings the parts related to video frame release decision making
in a single place and simplifies the calling side in
MediaCodecVideoRenderer.

PiperOrigin-RevId: 737941729
2025-03-18 03:46:13 -07:00
kimvde
bdc2216492 Skip looping sequences when computing composition duration
This fixes the following issue: in CompositionPlayer, if the
Composition was containing a looping sequence with duration larger than
all the other sequences, the Composition duration was set to the
looping sequence duration. This is incorrect because looping sequences
should be clipped to match the other sequences.

PiperOrigin-RevId: 737927987
2025-03-18 02:46:20 -07:00
shahddaghash
40ab0d40a1 Set LogSessionId on Transformer decoders and encoders
PiperOrigin-RevId: 737636636
2025-03-17 09:14:17 -07:00
ivanbuper
cc4ffbe8cf Enforce one non-looping sequence in Composition
Before this change, exporting a Composition with all looping sequences
would result in a runtime exception due to a divide-by-zero exception
in TransformerInternal.

PiperOrigin-RevId: 737568877
2025-03-17 04:45:45 -07:00
ivanbuper
d777a11840 Move createCodecProfileLevel() to MediaCodecUtil and simplify usages
This is a non-functional change.

PiperOrigin-RevId: 736534306
2025-03-13 09:34:10 -07:00
kimvde
03892cc1b5 VideoSink: merge setStreamStartPositionUs and onInputStreamChanged
setStreamStartPositionUs and onInputStreamChanged should both be called
when the stream changes.

PiperOrigin-RevId: 736121598
2025-03-12 07:17:41 -07:00
shahddaghash
1918a256cc Add a test for playing CompositionPlayer again after playback ends
A problem was discovered in Composition demo app where after playing a composition using CompositionPlayer, pressing the "play" button again would not restart the playback. This issue has been resolved, just adding a test to make sure we're not missing any regressions.

PiperOrigin-RevId: 735793488
2025-03-11 09:59:53 -07:00
michaelkatz
2729dbb8a9 Limit dynamic scheduling interval by the audio track buffer size
In certain bluetooth playback scenarios, it was found that the delta of audio duration written by the AudioSink from the current playback position was greater than the size of the audio track buffer, causing underruns.

The solution is to utilize the audio track buffer size as an upper limit for an audio renderer's getDurationToProgress.

PiperOrigin-RevId: 735761604
2025-03-11 08:16:22 -07:00
claincly
44b3a43652 Use VideoGraph as top level component
...and reduce the number of classes/interfaces

PiperOrigin-RevId: 735451687
2025-03-10 11:30:20 -07:00
kimvde
ce59680d0f Always call onProcessedStreamChanged for first stream for effects
Before this CL:
- MediaCodecVideoRenderer.onProcessedStreamChanged was called for the
  first stream in CompositionPlayer.
- MediaCodecVideoRenderer.onProcessedStreamChanged was NOT called for
  the first stream in ExoPlayer.setVideoEffects.

This discrepancy was adding some complexity to the code, making it less robust.

PiperOrigin-RevId: 735439320
2025-03-10 10:58:23 -07:00
shahddaghash
a110b02142 Move TestTransformerBuilder to test-util
This improves consistency by grouping it with other test builders and resolves a missing build rule, ensuring developers can properly use it for testing.

PiperOrigin-RevId: 735396181
2025-03-10 08:59:54 -07:00
sheenachhabra
66995a8816 Add checkNotNull for all getVideoCapabilities and similar calls
PiperOrigin-RevId: 735358217
2025-03-10 06:50:54 -07:00