775 Commits

Author SHA1 Message Date
samrobinson
3ac82b1d62 Add test cases for mixing to mono.
PiperOrigin-RevId: 517127021
2023-03-16 16:27:41 +00:00
huangdarwin
e6343a34b8 Test: Move 4k60 test file to local from remote.
This test file used to be remote, which led to various errors in loading the file or timing out during the muxer or overall transformer.

export4k60 test timing on Pixel 7:

|Condition                 |elapsedTimeMs|AS Test "Duration" (s)|
|--------------------------|-------------|----------------------|
|local file (this CL)      |4253         |14                    |
|Remote file, googleguest  |5510         |17                    |
|Remote file, 4G           |11423        |25                    |

PiperOrigin-RevId: 516815462
2023-03-16 15:43:35 +00:00
kimvde
df724b517e Add a way in the SampleConsumer to indicate that it couldn't consume
- To support looping EditedMediaItemSequences, we need a way to tell the
AssetLoader that a sample couldn't be consumed and that it should retry
later. This is necessary in case we don't know yet whether the looping
sequence should load more samples because the other sequences haven't
made sufficient progress yet.
- The decision on whether to consume a sample is based on its timestamp
so it needs to be available.

PiperOrigin-RevId: 516546026
2023-03-14 16:49:47 +00:00
samrobinson
57ce705760 Move RepeatedTranscodeTest to mh/analysis.
This test is not run on emulator, triage or postsubmit, and does not
provide value to the library in it's current state. It could be
deleted, however it still may have use for analysis of exported files
on MH.

PiperOrigin-RevId: 516491160
2023-03-14 16:01:15 +00:00
sheenachhabra
d579ccfd51 Remove setLocation() method from Muxer interface
PiperOrigin-RevId: 516314175
2023-03-14 09:24:06 +00:00
huangdarwin
6fd6781b8d Effect: Implement TimestampWrapper.
To allow applying an effect only on a range of timestamps.

PiperOrigin-RevId: 515615662
2023-03-14 08:59:53 +00:00
samrobinson
c7350f368f Remove PCM encoding from ExportResult.
PCM encoding is a property of decoded audio, however Transformer does
not output decoded audio.

PiperOrigin-RevId: 515580886
2023-03-14 08:55:54 +00:00
kimvde
c109e396ed Add test for audio and video from different sources
PiperOrigin-RevId: 515379858
2023-03-14 08:51:52 +00:00
sheenachhabra
f69160079c Add support for writing MP4 location data using media muxer
PiperOrigin-RevId: 515297752
2023-03-14 08:43:33 +00:00
kimvde
3a11c638a5 Split TransformerEndToEndTest
Split test/ TransformerEndToEndTest into SingleMediaItemEndToEndTest and
SingleSequenceEndToEndTest to reduce the file size and split the tests
by category.

PiperOrigin-RevId: 515039502
2023-03-14 08:19:19 +00:00
kimvde
2d97d3a571 Make sure errors thrown in the PlayerListener are not swallowed
PiperOrigin-RevId: 515037497
2023-03-14 08:15:19 +00:00
kimvde
4b97913e2e Remove VideoSamplePipeline unused param
PiperOrigin-RevId: 515002313
2023-03-14 07:55:26 +00:00
samrobinson
73e4348411 Verify the audio properties being used within Transformer.
Add TransformerAudioEndToEndTest to emulated tests.

PiperOrigin-RevId: 514991434
2023-03-14 07:51:18 +00:00
kimvde
87d1c3e624 Fix threading issueis in SequenceAssetLoaderListener
Callbacks onTrackCount and onTrackAdded can be called simultaneously
from different threads.

Before this fix, it was possible for the MuxerWrapper and
FallbackListener track count to never be set, or to be set
with incorrect values.

PiperOrigin-RevId: 514779719
2023-03-14 07:43:16 +00:00
tofunmi
82ab327e02 Update effect to take in and use a GlObjectsProvider
PiperOrigin-RevId: 514744747
2023-03-14 07:34:53 +00:00
samrobinson
83192e870c Fix AudioMixer javadoc.
PiperOrigin-RevId: 514699994
2023-03-07 15:02:21 +00:00
samrobinson
2d22604efe Pass Metadata to Muxer when adding a track.
PiperOrigin-RevId: 514575400
2023-03-07 12:53:00 +00:00
kimvde
ce9fa88c0d Rename CompositeAssetLoader to SequenceAssetLoader
This clarifies that a SequenceAssetLoader loads data corresponding to an
EditedMediaItemSequence.

PiperOrigin-RevId: 514442681
2023-03-07 12:44:22 +00:00
tofunmi
f94d3cf6c4 ImageAssetLoaderTest minor fixes
PiperOrigin-RevId: 514436359
2023-03-07 12:40:08 +00:00
tofunmi
6188952682 Add ImageAssetLoader tests.
PiperOrigin-RevId: 514431184
2023-03-07 12:35:55 +00:00
kimvde
55f2d1a5e2 Make onOutputFormat nullable
- This is to make sure we know about all the tracks before initializing
the SamplePipelines. This allows to set the muxer and the fallback
listener track count before the SamplePipelines are built.
- As a result, the test files had to be updated because the order in
which the tracks are written has changed.
- The ImageAssetLoader also had to be updated to call onOutputFormat
repeatedly until it returns a non-null SampleConsumer.
- Also fix the trackCount sent to the muxer and fallback listener. The
correct track count can be computed now that we know about all the
tracks before building the SamplePipelines.

PiperOrigin-RevId: 514426123
2023-03-07 12:31:43 +00:00
tofunmi
e8c86e1bae Remove effect/SimpleBitmapLoader & replace with DataSourceBitmapLoader
PiperOrigin-RevId: 513824487
2023-03-07 12:01:50 +00:00
kimvde
db540a02b7 Make sure that tracks are transcoded/transmuxed consistently
All audio tracks should either all be transcoded or all be transmuxed.
Same for video tracks.

To achieve this, simplify the behaviour of transmuxAudio/Video.

PiperOrigin-RevId: 513809287
2023-03-07 11:57:35 +00:00
samrobinson
93b9cc731d Provide TransformerInternal with the AssetLoader output format.
TI can use this Format as part of creating the SamplePipelines.

PiperOrigin-RevId: 513777622
2023-03-07 11:53:03 +00:00
andrewlewis
9f0a6f1619 Fix typo
PiperOrigin-RevId: 513529059
2023-03-02 17:35:30 +00:00
samrobinson
8642594379 Add getNormalizedTrackType, mapping image to video track type.
PiperOrigin-RevId: 513514142
2023-03-02 17:22:29 +00:00
samrobinson
5438e6cdb4 Only init SpeedChangingAudioProcessor if Format.Metadata not null.
If the Metadata passed to SegmentSpeedProvider is null, then the
SegmentSpeedProvider will always return 1f from getSpeed.

Initializing a SpeedChangingAudioProcessor requires a SpeedProvider.
Once configured,this audioProcessor is always active, so buffers are
passed through it. Because getSpeed is always 1, the processor performs
a no-op, but still has to do a buffer copy for each buffer.

By not initializing the audio processor when metadata is null, this
copy can be skipped and the audio pipeline is more performant.

Note: This change does not affect the multiple media-item case, which
is not supported with speed changes, as per Transformer API
documentation.
PiperOrigin-RevId: 513261811
2023-03-02 08:13:34 +00:00
ibaker
ef5a1ce932 Remove @see tags with <a> tags
These are not supported by Dackka

#minor-release

PiperOrigin-RevId: 513176533
2023-03-01 18:02:43 +00:00
andrewlewis
3634e552e6 Don't log removed encoders
We shouldn't have this logging unless we really need it to debug
a specific problem, as it can be noisy (even at debug level).

PiperOrigin-RevId: 512904412
2023-03-01 17:58:46 +00:00
kimvde
128d8ff036 Handle multiple sequences in a Composition
This lays the groundwork for full multi-asset, and more particularly for
adding looping background audio.

PiperOrigin-RevId: 512887888
2023-03-01 17:42:50 +00:00
claincly
b0659e12f0 Allow video format change.
Uses the first mediaItem's format as the output format.

If there is `Presentation` supplied in the `Composition.effects`, add it as the
last effect of the first EditedMediaItem.

PiperOrigin-RevId: 512082659
2023-02-27 18:38:52 +00:00
kimvde
e64f70effc Clarify some threading requirements.
PiperOrigin-RevId: 512079471
2023-02-27 18:34:21 +00:00
kimvde
2b02eb8c62 Update experimentalSetForceAudioTrack Javadoc to throw
This is consistent with the code behaviour.

PiperOrigin-RevId: 512073465
2023-02-27 18:30:07 +00:00
kimvde
be014f30c3 Small fixes in CompositeAssetLoader
PiperOrigin-RevId: 512038052
2023-02-27 18:12:45 +00:00
samrobinson
9353f18ea0 Fix broken test due to file not found.
PiperOrigin-RevId: 512012099
2023-02-27 18:03:41 +00:00
kimvde
3f445a25d8 Throw when the video track disappears during export
Throw when the output has a video track but the current MediaItem in
the sequence doesn't have any video.

PiperOrigin-RevId: 512004463
2023-02-27 17:59:17 +00:00
samrobinson
511ad57de6 Propagate PCM encoding passed to AudioSamplePipeline.
PiperOrigin-RevId: 511810757
2023-02-24 11:29:35 +00:00
huangdarwin
285d9c9fe5 Test: Add format codec info and update HDR10 file to 720p.
Add format codec info, which can make test skipping checks more similar to the
actual Transformer decoder checks.

Also for the test file, the actual format was 720p, but somehow the file name and
media metadata indicated 1080p. This format mismatch led to some decoding errors,
so fix the format (and associated errors). This also allows us to remove the
exception catch in ForceInterpretHdrVideoAsSdrTest, which was included due to
errors from the incorrect format.

PiperOrigin-RevId: 511809507
2023-02-24 11:26:05 +00:00
kimvde
41a6d4aafb Clarify slow mo flattening Javadoc
PiperOrigin-RevId: 511778088
2023-02-24 11:18:50 +00:00
kimvde
50f5e34583 Improve shouldTranscode() logic
PiperOrigin-RevId: 511767953
2023-02-24 11:11:40 +00:00
kimvde
c5c171beba Add unit tests for audio track dis(appearing) during export
PiperOrigin-RevId: 511764841
2023-02-24 11:08:07 +00:00
kimvde
4c9b8cb694 Document that slow mo flattening is only supported for single-asset
PiperOrigin-RevId: 511757712
2023-02-24 11:04:33 +00:00
kimvde
0205e75ff8 Split transmux into transmuxAudio/Video
- Split the transmux setting into transmuxAudio and transmuxVideo. This
  is more flexible for apps and will also be useful for unit testing
  (particularly as we can't test video transcoding on Robolectric at the
  moment).
- Move these settings to Composition. It makes sense for these settings
  to be next to forceAudioTrack. Apps may also want to set these
  settings based on the current Composition's MediaItems.
- Add a Composition.Builder because Composition now contains a few
  optional fields.

PiperOrigin-RevId: 511708618
2023-02-24 11:01:03 +00:00
samrobinson
7f3eb113e5 Fix disallowed tag.
If ChannelMixingMatrix is public, this causes code to not build.

PiperOrigin-RevId: 511500694
2023-02-24 10:57:22 +00:00
kimvde
379c3e1eb2 Restrict SampleConsumer and OnMediaItemChanged threading
This is more future-proof because it is likely to simplify the upcoming
changes to the sample pipelines.

PiperOrigin-RevId: 511492014
2023-02-24 10:53:53 +00:00
samrobinson
9a8026f27e Only log list of removed encoders if not empty.
Logcat had the following lines, with no other information.
```
DefaultEncoderFactory: Encoders removed for resolution:
DefaultEncoderFactory: Encoders removed for bitrate:
DefaultEncoderFactory: Encoders removed for bitrate mode:
```
PiperOrigin-RevId: 511470231
2023-02-24 10:50:20 +00:00
kimvde
12a6183312 Make sure that VideoSamplePipeline is picked for image input
PiperOrigin-RevId: 511449658
2023-02-24 10:46:49 +00:00
kimvde
948ccfe2a6 Document that the output file is not deleted in case of error/cancel
PiperOrigin-RevId: 511189130
2023-02-24 10:36:05 +00:00
huangdarwin
7e377854a3 MediaFormatUtil: Reduce Color API 29 restriction to API 24.
Implement getMediaFormatInteger, a helper method simulating mediaformat.getInteger(name, defaultValue).

This reduces the API 29 restriction from MediaFormatUtil.getColorInfo to API 24, in
particular removing the method-based restriction to a constant-based restriction,
so that we can reduce usage of the API 29 class.

This also allows us to slightly simplify prior use-cases where we'd check
containsKey and getInteger to have a default value.

PiperOrigin-RevId: 511184301
2023-02-24 10:32:22 +00:00
kimvde
f4766ee4db Fix clipping in AudioSamplePipeline
When clipping a MediaItem with start time > 0, the audio was ending
before the video. This is because:
- Audio timestamps are computed based on the sample sizes, with a start
  time set to streamOffsetUs (i.e. the streamStartPositionUs is not
  taken into account).
- The SamplePipeline was subtracting streamStartPositionUs from the
  timestamps before sending the samples to the muxer.
- As a result, the audio timestamps were shifted by
  streamStartPositionUs, while they should be shifter by streamOffsetUs.

PiperOrigin-RevId: 511175923
2023-02-24 10:28:53 +00:00