1521 Commits

Author SHA1 Message Date
dancho
d2fb779929 Ignore flaky preview tests
Tests are flaky because shader program creation is slow

PiperOrigin-RevId: 689420905
2024-10-24 10:04:57 -07:00
dancho
94e37ca7d2 Fix flaky test
* Use asset without audio for independent video time progress
* Use longer item duration to avoid ExoPlayer STATE_READY workaround
  (see https://github.com/google/ExoPlayer/issues/1874)

PiperOrigin-RevId: 689409427
2024-10-24 09:29:35 -07:00
kimvde
be8c58d51e Use Format object in VideoFrameProcessor
This is to use an existing media3 object rather than creating a new one.

PiperOrigin-RevId: 688481323
2024-10-22 03:46:03 -07:00
shahddaghash
320cbc09f4 Remove deprecated AudioMixer.create() method
Use `DefaultAudioMixer.Factory().create()` instead.

PiperOrigin-RevId: 688167322
2024-10-21 09:39:34 -07:00
ibaker
b04b37074b Add video language to DefaultTrackSelector
PiperOrigin-RevId: 688155680
2024-10-21 09:02:19 -07:00
claincly
73790cf2a4 Don't optimize for trim if the media is not clipped
PiperOrigin-RevId: 688120868
2024-10-21 06:58:59 -07:00
claincly
2a49ffcb23 Make OverlaySettings dynamic
PiperOrigin-RevId: 687269731
2024-10-18 05:43:33 -07:00
shahddaghash
2e61c93dba Remove deprecated DefaultEncoderFactory constructors.
Use `DefaultEncoderFactory.Builder` instead.

PiperOrigin-RevId: 686821388
2024-10-17 01:53:23 -07:00
kimvde
31ef7ff088 Clarify Javadoc of EditedMediaItem.Builder.setDurationUs
PiperOrigin-RevId: 686521901
2024-10-16 08:48:10 -07:00
kimvde
9adb3aaf41 Transformer: add an entry point to disable automatic rotation
PiperOrigin-RevId: 686067527
2024-10-15 05:40:41 -07:00
sheenachhabra
407bd49ed5 Skip audio encoding bitrate setting test on API <= 23
The encoder output format on API 23 does not seem to contain bitrate,
hence the test fails.

PiperOrigin-RevId: 686047480
2024-10-15 04:15:38 -07:00
Googler
0100f1d902 Boxes: Add edit list box.
PiperOrigin-RevId: 685974308
2024-10-14 23:26:08 -07:00
claincly
17c0ff8ba8 Log warnings when Transformer sees unsupported track type
PiperOrigin-RevId: 685649866
2024-10-14 03:54:52 -07:00
sheenachhabra
a0ccd46653 Skip AACObjectHE encoding on faulty devices
These devices claim to have the AACObjectHE profile but the profile never gets applied.

PiperOrigin-RevId: 684786157
2024-10-11 04:18:52 -07:00
ibaker
2c46cea088 Use RoundingMode.DOWN in Util.scaleLargeTimestamp and friends
The implementation of these methods was updated from direct java integer
arithmetic in 885ddb167e.
In this change, `RoundingMode.FLOOR` was used to try and maintain
compatibility with java integer division. This was incorrect, because
java integer division uses `DOWN` (i.e. towards zero), rather than
`FLOOR` (i.e. towards negative infinity) semantics.

This change fixes the compatibility.

The dump file changes in this CL relate to tests that exercise edit
list behaviour. This involves manipulating negative timestamps, which
explains why they are impacted by this change.

PiperOrigin-RevId: 684013175
2024-10-09 06:24:26 -07:00
kimvde
52f08d46c2 Add motion photo support to Transformer
PiperOrigin-RevId: 683540867
2024-10-08 02:57:52 -07:00
ivanbuper
af922fbcb0 Fix truncation error accumulation on Sonic's time stretching algorithm
This CL also fixes EOS handling to account for not-yet-copied samples in
`remainingInputToCopyFrameCount`, which would throw off the final output
sample count calculation.

For testing, we allow a tolerance of 0.000017% drift between expected
and actual number of output samples. The value was obtained from running
100 iterations of `timeStretching_returnsExpectedNumberOfSamples()` and
calculating the average delta percentage between expected and actual
number of output samples. Roughly, this means a tolerance of 40 samples
on a 90 min mono stream @48KHz.

PiperOrigin-RevId: 683133461
2024-10-07 04:59:54 -07:00
dancho
7b08bedf2c Do not force EOS when decoder has produced all frames
The workaround in ExternalTextureManager.forceSignalEndOfStream
was being applied even when the decoder did output all frames, and only
the GL pipeline was slow.

This change ensures that workaround is not applied when the decoder
has already produced all output frames.

PiperOrigin-RevId: 680471587
2024-09-30 01:50:10 -07:00
claincly
e0e9f5b057 Increase the maxImage on ImageReader, hoping less flaky tests
PiperOrigin-RevId: 679528425
2024-09-27 04:03:15 -07:00
dancho
b9aed0a937 Have AndroidTestUtil.canEncode mirror VideoSampleExporter
Some devices can encode portrait 720x1080 but not landscape
1080x720. But VideoSampleExporter always prefers encoding
landscape. Have `assumeFormatsSupported` mirror sample exporter
logic more closely

PiperOrigin-RevId: 679495210
2024-09-27 02:01:05 -07:00
samrobinson
45c400c7b5 Clarify compositionPlayer audio test naming.
PiperOrigin-RevId: 679142528
2024-09-26 07:37:52 -07:00
sheenachhabra
b6192f7a39 Add AudioEncoderSettings to Transformer
It allows clients to specify the audio encoding profile and bitrate.
This is similar to VideoEncoderSettings.

PiperOrigin-RevId: 679131963
2024-09-26 06:59:53 -07:00
claincly
50c879ee21 Remove stale comment
PiperOrigin-RevId: 679082395
2024-09-26 03:55:24 -07:00
Googler
020ce7765c Reduce rounding error and stts table entries.
To avoid rounding errors, set the `Rounding mode` of the `uvFromVu` and `vuFromUs` results to `HALF_UP`. This `Rounding mode` rounds numbers towards the "nearest neighbor" unless both neighbors are equidistant, in which case round up.

PiperOrigin-RevId: 679003943
2024-09-25 23:19:20 -07:00
claincly
0ea63e3fa6 Disable video only on the sequence that has items that disable video
This means we need a custom track selector for each SequencePlayer.

PiperOrigin-RevId: 678779216
2024-09-25 11:28:26 -07:00
kimvde
3ec9c99644 Document that the MediaItem's image duration should be set for images
PiperOrigin-RevId: 678702334
2024-09-25 08:04:31 -07:00
kimvde
2dc32360d6 Remove usages of deprecated DefaultDecoderFactory constructor
The constructor was deprecated but usages were not migrated.

PiperOrigin-RevId: 678683678
2024-09-25 07:01:14 -07:00
samrobinson
fa04386863 Enable multi sequence audio preview test.
PiperOrigin-RevId: 678667829
2024-09-25 06:03:23 -07:00
samrobinson
5ab72a3938 Use constants in dump file names for playback test.
PiperOrigin-RevId: 678664730
2024-09-25 05:51:06 -07:00
kimvde
eaec7a4a61 Set MediaItem's image duration for image export
This field will become mandatory

PiperOrigin-RevId: 678656879
2024-09-25 05:23:27 -07:00
kimvde
f83d2b1392 Stop calling setDurationUs when image duration already set
The EditedMediaItem's duration is set to the MediaItem's image duration
by default.

PiperOrigin-RevId: 678597324
2024-09-25 02:02:30 -07:00
samrobinson
fc07ce056a Add safer gap based checks to Transformer API boundary points.
PiperOrigin-RevId: 678278666
2024-09-24 09:23:01 -07:00
samrobinson
076eea283e Adding API and internal export support for audio gaps in sequences.
PiperOrigin-RevId: 678235289
2024-09-24 07:14:59 -07:00
kimvde
d8dc513431 Apply video composition effects in preview
PiperOrigin-RevId: 678207818
2024-09-24 05:46:33 -07:00
kimvde
c19d910f6b Fix EffectPlaybackPixelTest
The tests using createTimestampOverlay() were passing even if the effect
was removed, because the overlay was too small.

PiperOrigin-RevId: 678169395
2024-09-24 03:24:49 -07:00
samrobinson
b4436c523c Migrate Media3 EditedMediaItemSequence usages to the Builder.
Removed the unnecessary wrapping of items in an ImmutableList.

PiperOrigin-RevId: 677796662
2024-09-23 08:06:55 -07:00
ivanbuper
3d3ec85c12 Setup basic testing for Sonic and assert expected sample count drift
This CL adds `SonicTest` and `RandomParameterizedSonicTest` as
initial basic unit testing for `Sonic.java`. The tested scenarios
do not necessarily verify a correct implementation of Sonic, but rather
hope to catch any behaviour change from the current implementation.

The change includes a small fix for a lossy simplification and also
checks whether the output sample count matches the expected drift from
the truncation accumulation error present in Sonic's resampler. This is
important as pre-work for fixing issues with unexpected durations within
`SonicAudioProcessor` and `SpeedChangingAudioProcessor` that cause AV
sync issues for speed changing effects.

This is a partial roll forward of e88d6fe459, which was rolled back in
873d485056.

PiperOrigin-RevId: 677756854
2024-09-23 05:53:58 -07:00
kimvde
be4d31ba87 Prevent ExoPlayer dropping frames in EffectPlaybackPixelTest
Some tests check all the output frames and fail if there are frames
missing.

PiperOrigin-RevId: 677676143
2024-09-23 01:12:40 -07:00
samrobinson
3c5e764b86 Fix CapturingMuxer writing of final PCM audio sample to dump file.
Last buffer was not flipped, so was writing the garbage data between
limit and capacity, rather than the actual data between position and
limit.

As a result, all PCM audio dump files need updating.

PiperOrigin-RevId: 676452990
2024-09-19 09:52:12 -07:00
samrobinson
75c7ee79d5 Add EditedMediaItemSequence.Builder.
PiperOrigin-RevId: 676422122
2024-09-19 08:23:37 -07:00
kimvde
980f24d906 Avoid ImageReader frame drops in EffectPlaybackPixelTest
If this doesn't work, we could make this test resilient to frame drops
by only comparing the frames that weren't dropped.

PiperOrigin-RevId: 676294944
2024-09-19 00:18:46 -07:00
kimvde
caf70e54db Fix outdated Transformer Javadoc
PiperOrigin-RevId: 675942348
2024-09-18 04:56:51 -07:00
ibaker
3e8ecbf564 Remove @DoNotInline annotations
This is no longer needed now our `compileSdk` implies a new-enough AGP
which does this out-lining automatically via R8. See also
https://issuetracker.google.com/345472586#comment7

There's no plan to remove the `ApiXXX` classes, but no new ones need
to be added.

PiperOrigin-RevId: 675940634
2024-09-18 04:47:39 -07:00
kimvde
2951a2599c Apply all video Composition effects to single sequence exports
PiperOrigin-RevId: 675933601
2024-09-18 04:18:55 -07:00
claincly
fd3d8e1782 Add RATE_UNSET option to encoder performance setting
This is to allow not setting the MediaFormat OPERATING_RATE and PRIORITY
altogether. The current behvaiour, if left the value `UNSET`, it'll apply the
our optimizations, but apps might want to disable this optimization.

PiperOrigin-RevId: 675923909
2024-09-18 03:40:13 -07:00
claincly
69da26935e Fix logic that detects the last media item in Sequence
After the change in a879bc2154, the Sequence won't have repeated
EditedMediaItems. Thus if the sequence is looping, the last EditedMediaItems
in the Sequence object might not corresponds to the last item in the "logical"
sequence.

PiperOrigin-RevId: 675912197
2024-09-18 02:54:11 -07:00
sheenachhabra
acb8e71c6e Set hasMuxedTimestampZero to true only when its muxed
The muxer might not have accepted the first sample, if it
is waiting for audio track.

This bug causes issue when

1. VideoSampleExporter gives first sample (timestamp = 0) to the muxer.
2. Muxer does not write it because its waiting for audio track.
3. The video pipleline has processed all the sample and they are ready
to be consumed.
4. VideoSampleExporter fetches the next available sample from encoder (which is still with timestamp = 0) but it changes its timestamp to last timestamp because VideoSampleExporter thinks it has muxed the sample at timestamp zero, but in reality it hasn't. This is because the flag `hasMuxedTimestampZero` is set when queueing the input, rather than actually muxing the input.

This scenario can happen when video is processed much faster than
the audio.

PiperOrigin-RevId: 675565603
2024-09-17 07:53:22 -07:00
ivanbuper
873d485056 Rollback of e88d6fe459
PiperOrigin-RevId: 675525508
2024-09-17 05:25:07 -07:00
kimvde
0ea229d795 Remove limit of 2 sequences in CompositionPlayer
The player supports more than 2 audio sequences

PiperOrigin-RevId: 675493637
2024-09-17 03:20:28 -07:00
sheenachhabra
47d45a82ca Change the default value of lastSampleDurationBehavior
to
LAST_SAMPLE_DURATION_BEHAVIOR_SET_FROM_END_OF_STREAM_BUFFER_OR_DUPLICATE_PREVIOUS

This CL also combines LAST_SAMPLE_DURATION_BEHAVIOR_SET_FROM_END_OF_STREAM_BUFFER
and LAST_SAMPLE_DURATION_BEHAVIOR_DUPLICATE_PREVIOUS.

The reason for combining the two enums is that, when the option
to use END_OF_STREAM_BUFFER is selected and if the EOS buffer is
not provided then the muxer anyways fallbacks to duplicate
duration behavior.

The last sample with 0 durations seems less useful so
change the default behavior to non-zero duration.
This will also match the behavior with MediaMuxer.

PiperOrigin-RevId: 675189932
2024-09-16 10:02:10 -07:00