In a image sequence, seek to after the image duration should render the final
image frame. But currently it hangs as ConstantRateTimestampIterator doesn't
output any timestamp in this case
PiperOrigin-RevId: 724295475
This avoids that these settings have to be resolved inline,
potentially blocking the main thread. They can be resolved at
the time of track selection on a background thread instead.
As a side effect, we can also remove the context parameter from
the Builder. Having the Context in the Builder is also a bad sign
in the first place because it implies the potentially blocking
calls can happen.
PiperOrigin-RevId: 720523139
This is a common pattern in media3 libraries where tasks are handled on a
background thread, but the calling thread sees an immediate state update
with a best-guess placeholder. This makes the integration for the caller
very easy as the API surface appears to be synchronous.
This util is a helper class to handle this logic and test it separately.
PiperOrigin-RevId: 716233966
When the current MediaMetadata is automatically derived from the
MediaItem and Tracks, the result may change when state.buildUpon()
is used and new input data is provided (e.g. new current index).
To avoid reusing the previously derived MediaMetadata, we need to
reset it to null and re-evaluate it on every call to build()
Issue: androidx/media#1940
PiperOrigin-RevId: 714021424
In some cases the position is supplied from a "live" source
that keeps changing its value at repeated calls. This typically
happens when forwarding to another backend with unpredictable
position values. The problem is that as soon as the backend
system has a position discontinuity, any previously created
PositionSupplier now returns a position that no longer makes
sense in the context of the State object it belongs to.
ForwardingSimpleBasePlayer already works around this issue by
having a util class to disconnect these live sources. This
change moves the same util to SimpleBasePlayer itself so it
can be reused by other implementations.
PiperOrigin-RevId: 713658046
This change simplifies SpeedChangingAudioProcessor by removing unneeded
heuristics and synchronization added as workarounds to estimate input
and output frame counts.
The synchronization between the video processing and audio processing
threads cannot be completely removed yet because the static methods
depend on the input sample rate for the calculations. However, once
`SpeedChangingAudioProcessor` has been configured, then
`#getSpeedAdjustedTimeAsync()` should invoke the callback immediately.
PiperOrigin-RevId: 712893246
For the [0.5; 1) speed range, the combination of having a "slow down"
speed (i.e. more output frames than input frames), and Sonic potentially
needing to copy more input frames that are available in the input buffer
can lead to an unexpected underflow.
Specifically, the underflow happens in Sonic#queueEndOfStream() when the
following conditions are met (skipping some minor ones):
1. `inputFrameCount < remainingInputToCopyFrameCount`
2. `0.5f <= speed < 1`.
3. `outputFrameCount <
(inputFrameCount / remainingInputToCopyFrameCount) / 2`.
This underflow caused `SonicAudioProcessor#isEnded()` to return a false
negative (because `getOutputSize() != 0`), which would stall the
`DefaultAudioSink` waiting for processing to end after EOS.
In practical terms, the underflow is relatively easy to reproduce if we
consume all of Sonic's output and then immediately queue EOS without
queueing any more input in between. This should cause both
`inputFrameCount` and `outputFrameCount` to drop to 0.
PiperOrigin-RevId: 711773565
The static method can estimate the number of input frames needed to get
a given number of output frames with a given Sonic configuration.
This CL is prework to remove the dependency of
`SpeedChangingAudioProcessor#getMediaDurationUs()` on non-static output
based heuristics and simplify `SpeedChangingAudioProcessor`.
PiperOrigin-RevId: 711446999
`Parcelable` is not safe for IPCs between binaries with potentially
different class definitions (e.g. two apps built from different versions
of media3).
If we get a use case to bundle metadata, then the `Metadata` and
`Metadata.Entry` classes needs to provide a `toBundle()` method.
PiperOrigin-RevId: 706678892
Metadata does not provide a `toBundle` method. It implements
`Parcelable` which is not safe for IPCs between binaries with
potentially different class definitions (e.g. two apps built from
different versions of media3).
If we get a use case to bundle metadata as well, then the `Metadata`
class needs to provide a `toBundle()` method.
PiperOrigin-RevId: 705920231
This code previously passed null or 'default' for every parameter.
Now it tries to mock non-final types, and recursively constructor final
types if possible, eventually giving up and passing null instead.
The previous null-passing behaviour led to a quite confusing failure in
`ForwardingPlayer.addListener` when writing 25c927e9f3 due to an NPE
when trying to compare parameter equality in `Mockito.verify` [1].
With this change, the failure is much clearer [2].
There's a relatively simple case this code still doesn't handle: A final
type like `PlaybackParameters` where the constructor parameters **have**
to be non-default primitives (greater than zero in that case).
-------
[1]
```
java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at androidx.media3.test.utils.TestUtil.assertForwardingClassForwardsAllMethodsExcept(TestUtil.java:687)
at androidx.media3.common.ForwardingPlayerTest.forwardingPlayer_forwardsAllPlayerMethods(ForwardingPlayerTest.java:110)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.robolectric.RobolectricTestRunner$HelperTestRunner$1.evaluate(RobolectricTestRunner.java:588)
at org.robolectric.internal.SandboxTestRunner$2.lambda$evaluate$2(SandboxTestRunner.java:290)
at org.robolectric.internal.bytecode.Sandbox.lambda$runOnMainThread$0(Sandbox.java:101)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.lang.NullPointerException: Cannot invoke "Object.hashCode()" because "this.listener" is null
at androidx.media3.common.ForwardingPlayer$ForwardingListener.hashCode(ForwardingPlayer.java:1140)
at java.base/java.lang.Object.toString(Object.java:256)
at java.base/java.lang.String.valueOf(String.java:4220)
at org.mockito.internal.verification.argumentmatching.ArgumentMatchingTool.toStringEquals(ArgumentMatchingTool.java:54)
at org.mockito.internal.verification.argumentmatching.ArgumentMatchingTool.getSuspiciouslyNotMatchingArgsIndexes(ArgumentMatchingTool.java:36)
at org.mockito.internal.verification.checkers.MissingInvocationChecker.checkMissingInvocation(MissingInvocationChecker.java:45)
at org.mockito.internal.verification.Times.verify(Times.java:37)
at org.mockito.internal.verification.MockAwareVerificationMode.verify(MockAwareVerificationMode.java:30)
at org.mockito.internal.handler.MockHandlerImpl.handle(MockHandlerImpl.java:75)
at org.mockito.internal.handler.NullResultGuardian.handle(NullResultGuardian.java:29)
at org.mockito.internal.handler.InvocationNotifierHandler.handle(InvocationNotifierHandler.java:34)
at org.mockito.internal.creation.bytebuddy.MockMethodInterceptor.doIntercept(MockMethodInterceptor.java:82)
at org.mockito.internal.creation.bytebuddy.MockMethodInterceptor.doIntercept(MockMethodInterceptor.java:56)
at org.mockito.internal.creation.bytebuddy.MockMethodInterceptor$DispatcherDefaultingToRealMethod.interceptAbstract(MockMethodInterceptor.java:161)
at androidx.media3.common.Player$MockitoMock$1276619531.addListener(Unknown Source)
... 22 more
```
----
[2]
```
Argument(s) are different! Wanted:
player.addListener(
Mock for Listener, hashCode: 107929032
);
-> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Actual invocations have different arguments:
player.addListener(
androidx.media3.common.ForwardingPlayer$ForwardingListener@af47cec0
);
-> at androidx.media3.common.ForwardingPlayer.addListener(ForwardingPlayer.java:81)
```
PiperOrigin-RevId: 702378861
Add missing ones for `ForwardingExtractorInput` and `ForwardingSeekMap`.
`ForwardingTimeline` is a bit more fiddly, so it's left for a follow-up
change.
PiperOrigin-RevId: 701988492
This non-functional refactor inlines and simplifies the logic of
`BaseAudioProcessor` used by `SpeedChangingAudioProcessor`, and makes
the latter implement `AudioProcessor` directly.
`SpeedChangingAudioProcessor` acts as a wrapper over
`SonicAudioProcessor` and does not actually modify any samples,
so it had very little use for the affordances provided by
`BaseAudioProcessor`.
PiperOrigin-RevId: 698342962
SpeedChangingAudioProcessor is redundantly bypassing SonicAudioProcessor
when the speed is set to 1f. SpeedChangingAudioProcessor should not make
assumptions about the underlying audio processing and moreover should
not be bypassing any AudioProcessor based on parameter values. Default
parameter values are a valid state for an active AudioProcessor.
Sonic already handles the "default case" state by just copying the input
buffer onto the output buffer.
This CL also simplifies SonicAudioProcessor, which would mark itself as
inactive when configured with a valid set of default parameters. The API
contract for `isActive()` makes no mention about parameter state, which
makes changes in `isActive()` after applying new valid parameters quite
unintuitive.
PiperOrigin-RevId: 698000500
This method will replace the current estimation mechanism in place on
`getSpeedAdjustedTimeAsync()`, which relies on heuristics from previous
output rates and depends on the state of the `AudioProcessor`. The new
static method should be considerably more accurate.
PiperOrigin-RevId: 694607264
Also move the `lint.xml` config which disables the `NewApi` check from
`lib-session` to the existing top-level file, and limit it to cover all
Robolectric tests by path matching.
Follow-up to 76db936d68
PiperOrigin-RevId: 692913646
This CL adds utility methods to obtain sample-aligned timestamps from a
`SpeedProvider` to solve in a unified way the issues arising from
conversions between sample positions and microseconds.
The new utility methods will also help with the implementation of a
static method like `getDurationAfterProcessorApplied()` that will remove
the need for thread synchronization between the video and audio pipeline
for speed changing effects.
PiperOrigin-RevId: 692233318
This method allows `Sonic` to statically and accurately report the
expected number of output frames for any given parameter configuration.
This change is required prework for `SpeedChangingAudioProcessor` to
implement a similar static method and allow precise, non-blocking
timestamp adjustments for the experimental speed changing effect.
PiperOrigin-RevId: 690669627
This is prework for `Sonic` to provide a precise calculation of output
sample counts, and thus allow `SpeedChangingAudioProcessor` to replace
`getSpeedAdjustedTimeAsync()` with an accurate synchronous calculation.
This is a non functional change.
PiperOrigin-RevId: 690608520
`RandomParameterizedSpeedChangingAudioProcessorTest` follows the
structure of `RandomParameterizedSonicAudioProcessorTest` and will help
improve coverage and confidence in the output length of
`SpeedChangingAudioProcessor`.
This CL is prework for removing the synchronization between the video
pipeline and the audio pipeline for speed changing effects.
PiperOrigin-RevId: 688578597
This is prework for implementing
`RandomParameterizedSpeedChangingAudioProcessorTest`, which depends on
Sonic's resampling algorithm's behaviour.
This is a non-functional refactor.
PiperOrigin-RevId: 686874593
This CL is prework for implementing
`RandomParameterizedSpeedChangingAudioProcessorTest`, which will build
on logic present in `RandomParameterizedSonicTest`.
This is a non-functional change.
PiperOrigin-RevId: 684838017
Inconsistent rounding modes between `currentTimeUs` and
`bytesUntilNextSpeedChange` would cause `SpeedChangingAudioProcessor`
to miss calling `queueEndOfStream()` on `SonicAudioProcessor` on a speed
change, and thus the final output samples of that `SonicAudioProcessor`
"configuration" would be missed.
This change is also a partial revert of 971486f5f9, which fixed a hang
of `SpeedChangingAudioProcessor`, but introduced the dropped output
frames issue fixed in this CL (see b/372203420). To avoid reintroducing
the hang, we are now ignoring any mid-sample speed changes and will only
apply speed changes that are effective at a whole sample position.
PiperOrigin-RevId: 684824218
This helps avoid overflows in intermediate calculations.
Verified the value in the test using `BigInteger`:
```
jshell> BigInteger.valueOf(1L << 52).multiply(BigInteger.valueOf(90000)).divide(BigInteger.valueOf(1000000))
$3 ==> 405323966463344
```
Issue: androidx/media#1763
#cherrypick
PiperOrigin-RevId: 684028178
This CL also fixes EOS handling to account for not-yet-copied samples in
`remainingInputToCopyFrameCount`, which would throw off the final output
sample count calculation.
For testing, we allow a tolerance of 0.000017% drift between expected
and actual number of output samples. The value was obtained from running
100 iterations of `timeStretching_returnsExpectedNumberOfSamples()` and
calculating the average delta percentage between expected and actual
number of output samples. Roughly, this means a tolerance of 40 samples
on a 90 min mono stream @48KHz.
PiperOrigin-RevId: 683133461
The MetadataRenderer by default supports icy and vnd.dvb.ait content. Those tracks should therefore be set with the `C.TrackType` `TRACK_TYPE_METADATA` rather than `TRACK_TYPE_UNKNOWN`.
PiperOrigin-RevId: 679132680
This CL adds `SonicTest` and `RandomParameterizedSonicTest` as
initial basic unit testing for `Sonic.java`. The tested scenarios
do not necessarily verify a correct implementation of Sonic, but rather
hope to catch any behaviour change from the current implementation.
The change includes a small fix for a lossy simplification and also
checks whether the output sample count matches the expected drift from
the truncation accumulation error present in Sonic's resampler. This is
important as pre-work for fixing issues with unexpected durations within
`SonicAudioProcessor` and `SpeedChangingAudioProcessor` that cause AV
sync issues for speed changing effects.
This is a partial roll forward of e88d6fe459, which was rolled back in
873d485056.
PiperOrigin-RevId: 677756854
To support for 3gpp h263 codec in Mp4Muxer currently profile and level is hardcoded and provided to h263 box. Parse profile and level from MediaFormat and use those value to write h263 box.
PiperOrigin-RevId: 675004590
Sonic would accumulate truncation errors on float to int conversions
that caused the final output sample count to drift noticeably, by
hundreds of samples on streams of a few minutes of length. The fix now
keeps track of the truncation error and compensates for it.
Other small fixes include eliminating lossy operations (e.g. int
division) and using doubles instead of floats for resampling where
helpful.
This CL also introduces `SonicParameterizedTest`, which helps test
resampling on an arbitrary number of randomly generated parameters,
with random sample data. `SonicParameterizedTest` uses `BigDecimal`s
for calculating sample count values, as to avoid precision issues with
large sample counts.
PiperOrigin-RevId: 673852768
`Muxer` module needs to use this method, hence moved to common.
This CL also makes `getHevcProfileAndLevel` public because this is used
in `MediaCodecUtil`.
PiperOrigin-RevId: 671739166
These were missed when upgrading to both SDK 19 and 21, but they now
cause failures like:
```
Caused by: java.lang.RuntimeException: Failed to parse package buildout/intermediates/apk_for_local_test/debugUnitTest/packageDebugUnitTestForUnitTest/apk-for-local-test.ap_: buildout/intermediates/apk_for_local_test/debugUnitTest/packageDebugUnitTestForUnitTest/apk-for-local-test.ap_ (at Binary XML file line #20): Requires newer sdk version #21 (current version is #19)
at org.robolectric.shadows.ShadowPackageParser.callParsePackage(ShadowPackageParser.java:61)
... 20 more
```
#cherrypick
PiperOrigin-RevId: 670241471
