947 Commits

Author SHA1 Message Date
samrobinson
306ce6e3a1 Don't use transformationResult with timeout/unexpected exceptions.
PiperOrigin-RevId: 503187291
2023-01-23 12:31:21 +00:00
kimvde
2cb4e3d71e Remove usages of deprecated startTransformation
PiperOrigin-RevId: 503138745
2023-01-23 12:27:00 +00:00
kimvde
aa72b45cdf Make startTransformation methods take an EditedMediaItem
Usages of the deprecated methods in Transformer.Builder will be removed
in a follow-up CL.

PiperOrigin-RevId: 502889423
2023-01-23 12:24:44 +00:00
kimvde
d4db33a535 Avoid re-encoding if video effects are no-op
This is to avoid regressions introduced by removing the convenience
methods from TransformationRequest.

PiperOrigin-RevId: 502864512
2023-01-23 12:23:42 +00:00
samrobinson
6dc119a8ab Add ColorInfo to TransformationResult.
TESTED: Manually verified the ColorInfo is propagated when VSP is used.
PiperOrigin-RevId: 502843773
2023-01-23 12:21:20 +00:00
samrobinson
dfa98ae791 Generate silent audio if no audio track is available.
To always generate silent audio, `removeAudio(true)` can be used in conjunction.

PiperOrigin-RevId: 502814315
2023-01-18 12:03:59 +00:00
kimvde
b0f82767aa Add EditedMediaItem class
This class will be used in a follow-up CL.

PiperOrigin-RevId: 502615323
2023-01-18 12:03:12 +00:00
samrobinson
a3b0708d31 Add resolution fallback for 25% of requested.
If a device only supports 1920x1080 as maximum resolution, then adding
the 25% case ensures 8k is reduced correctly.

PiperOrigin-RevId: 502588364
2023-01-18 12:01:02 +00:00
samrobinson
d092b5261e Add a unit test to check 2/3 resolution supported.
PiperOrigin-RevId: 502573469
2023-01-18 12:00:08 +00:00
kimvde
574fea39b6 Add CompositeAssetLoader
This is an AssetLoader that wraps a sequence of AssetLoaders. It will
be used for constrained multi-asset.

This class can currently only concatenate media items with the exact
same format.

PiperOrigin-RevId: 502525796
2023-01-18 11:57:03 +00:00
kimvde
7803716a77 Remove setScale and setRotationDegrees from TransformationRequest
Also remove usages of TransformationRequest convenience methods
(setScale, setRotationDegrees and setResolution).

Some usages of setResolution can't be removed yet because they are used
for fallback.

PiperOrigin-RevId: 502415748
2023-01-17 02:14:20 +00:00
kimvde
15ba0c5fa6 Add the possility to shift frame timestamps in SampleConsumer
This is needed for constrained multi-asset to shift the timestamps of
the media items that are not the first in the sequence.

PiperOrigin-RevId: 502409923
2023-01-17 02:12:48 +00:00
samrobinson
1eeccb6c69 Move SonicAudioProcessor to media3.common module.
PiperOrigin-RevId: 501881646
2023-01-17 02:04:57 +00:00
claincly
a82fcdefcb Allow forcing duration in FrameworkMuxer.
PiperOrigin-RevId: 501865706
2023-01-17 02:03:24 +00:00
huangdarwin
d1e03a41ab HDR: Implement PQ to SDR tone-mapping.
Tested manually on the Pixel 7 and Samsung S10.

PiperOrigin-RevId: 501626354
2023-01-17 01:59:21 +00:00
huangdarwin
d300b37a55 Transformer: Clarify setResolution javadoc.
This class may be removed soon, but in the meantime clarify this method's
javadoc summary fragment to make it marginally clearer that this only affects
the "displayed" height instead of the "encoded" height.

PiperOrigin-RevId: 501582219
2023-01-17 01:57:38 +00:00
huangdarwin
e47307d95f Transformer: Simplify EncoderWrapper rotationDegrees logic.
Small refactoring change to simplify the logic a tiny bit by removing one
unnecessary variable. No functional change intended.

PiperOrigin-RevId: 501566533
2023-01-17 01:56:09 +00:00
samrobinson
d2c5a1bfb9 Move BaseAudioProcessor abstract class to common.
This is a prerequisite step to making the `ResamplingAudioProcessor` public.

PiperOrigin-RevId: 501545561
2023-01-17 01:55:29 +00:00
samrobinson
13537a170b Allow Muxer.writeSampleData to take @C.BufferFlag int flags.
PiperOrigin-RevId: 501314812
2023-01-17 01:52:27 +00:00
kimvde
8aaab01841 Remove setDecoderFactory from Transformer
It makes more sense to have it in the AssetLoader now that decoding is
done there, particularly as the decoder factory should rarely be
customized.

Also remove the setter from the AssetLoader.Factory because it's
uncommon to have setters in a factory.

PiperOrigin-RevId: 501244974
2023-01-17 01:49:36 +00:00
kimvde
63b8cae263 Add a queue at the start of the buffer SamplePipelines
This improves performance and makes the code more intuitive.

PiperOrigin-RevId: 501220234
2023-01-17 01:48:51 +00:00
kimvde
9880926dd5 Remove setters from AssetLoader.Factory
It's unusual to have setters on a Factory.

PiperOrigin-RevId: 501212264
2023-01-17 01:47:57 +00:00
steveanton
9a9da0ab8e Add an audio mixer that combines concurrent audio sources
Design document: go/me-android-audio-mixer

PiperOrigin-RevId: 501048758
2023-01-17 01:45:12 +00:00
samrobinson
316174b6fb Make EncoderBitrateProvider and implementations package private.
There is no API to allow applications to use these currently. Until
that exists, these classes do not need to be public.

PiperOrigin-RevId: 500792401
2023-01-10 18:38:53 +00:00
samrobinson
5a86644547 Add audio and video values from Format to TransformationResult.
PiperOrigin-RevId: 500789076
2023-01-10 18:38:11 +00:00
samrobinson
77f26264a3 Split IAE and ISE cases in MuxerWrapper and improve javadoc.
PiperOrigin-RevId: 500786191
2023-01-10 18:37:21 +00:00
samrobinson
ee5e3c0243 Improve docs and split audio/video order in TransformationResult.
PiperOrigin-RevId: 500776260
2023-01-10 18:36:43 +00:00
kimvde
b4f820d2af Rename SampleConsumer dequeueInputBuffer to getInputBuffer
This makes more sense as SampleConsumer.dequeueInputBuffer returns the
same buffer as long as it is not queued.

PiperOrigin-RevId: 500631982
2023-01-10 18:35:57 +00:00
kimvde
98bc817fe7 Fix sample interleaving
Whether to write a sample or not was based on the timestamp of the
previous sample, rather than the current sample.

PiperOrigin-RevId: 500195279
2023-01-10 18:35:11 +00:00
kimvde
b11fe97f66 Split ExoPlayerAssetLoaderRenderer
Split ExoPlayerAssetLoaderRenderer into audio and video renderers.

PiperOrigin-RevId: 500102256
2023-01-10 18:34:26 +00:00
samrobinson
f32b632b09 Refactor MuxerWrapper handling of track details.
This brings together the multiple details about a muxer track, and
reduces the need for additional variables and more complicated track
tracking.

PiperOrigin-RevId: 499872145
2023-01-10 18:31:40 +00:00
kimvde
cfc61655ce Rename the SamplePipelines
The old names are not really correct anymore because:
- The Audio/VideoTranscodingSamplePipelines do not decode anymore.
- The pipelines now mux the encoded data.

PiperOrigin-RevId: 499498446
2023-01-04 21:56:14 +00:00
samrobinson
37f8a0bb6b Improve encoder resolution capability checking.
Whilst testing fallback functionality, I found that we were
aggressively reducing the resolution if it was not supported. A quick
test found that we could reduce by a much smaller increments.
Performance wise it appears these checks are incredibly quick.

The code for checking supported sizes was duplicated, with one case
having a bug because of this duplication (2/3 case). This CL abstracts
this into a loop.

PiperOrigin-RevId: 499497646
2023-01-04 21:55:15 +00:00
kimvde
2eb255431f Merge BaseSamplePipeline and SamplePipeline
PiperOrigin-RevId: 499469006
2023-01-04 21:54:06 +00:00
kimvde
d4491427de Move video decoding to AssetLoader
PiperOrigin-RevId: 499454273
2023-01-04 21:50:58 +00:00
kimvde
2cda644915 Remove unnecessary exceptions in tests
PiperOrigin-RevId: 499160043
2023-01-04 21:47:57 +00:00
kimvde
bd0d61d80d Misc small fixes
- Improve AssetLoader Javadoc
- Fix Javadoc of sample pipelines
- Improve Javadoc of Transformer.startTransformation
- Move thread hops for Transformer listener callbacks to
  TransformerInternal

PiperOrigin-RevId: 499156604
2023-01-04 21:46:59 +00:00
samrobinson
4541db3541 Deprecated onTransformationError methods are not called.
When a listener method is deprecated, the new method should (by
default) called through to the deprecated one. This is because any
class that implements the method that is now deprecated needs to still
receive that callback.

It appears when onTransformationError(MediaItem, Exception) was
deprecated in favour of onTransformationError(MediaItem,
TransformationException), this deprecation was the wrong way round, and
the newer callback - onTransformationError(MediaItem,
TransformationResult, TransformationException) continued this mistake.

This CL now corrects this.

PiperOrigin-RevId: 498221504
2023-01-04 21:45:49 +00:00
kimvde
310434a8f9 Remove mentions of player in TransformationException
PiperOrigin-RevId: 497921432
2023-01-04 21:44:40 +00:00
samrobinson
88fd9e18f5 Simplify TransformerEndToEndTest and TransformerTestRunner.
PiperOrigin-RevId: 497335605
2023-01-04 21:42:37 +00:00
samrobinson
d09fdd1b52 Add a MuxerWrapper listener for events.
Events on the wrapper should be propagated to TransformerInternal as
soon as they occur, switching round the process so TransformerInternal
does not have to query MuxerWrapper.

This CL is a prerequisite for the child CL, where MuxerWrapper can
simplify the internal state and logic.

PiperOrigin-RevId: 497267202
2023-01-04 21:41:41 +00:00
tofunmi
283cbcb464 Specify units for VideoEncoderSettings.
PiperOrigin-RevId: 497113780
2023-01-04 21:37:18 +00:00
kimvde
e8572be6d5 Ensure AssetLoader callbacks are always called in the right order
Before, it was possible for onDurationUs() and onAllTracksRegistered()
to be called before onTrackAdded() because they are called from
different threads. onDurationUs() and onAllTracksRegistered() are called
from the Transformer internal thread, and onTrackAdded() is called from
the playback thread.

PiperOrigin-RevId: 497102556
2023-01-04 21:35:35 +00:00
kimvde
431108063c Specify the types of samples output by AssetLoader
This is useful if an app passes a custom AssetLoader that can only
output encoded or decoded samples.

PiperOrigin-RevId: 497097492
2023-01-04 21:34:25 +00:00
kimvde
f67849afa1 Add IntRange annotation to onTrackCount
PiperOrigin-RevId: 496942702
2023-01-04 21:28:49 +00:00
kimvde
5c4d85c2ed Change MuxerWrapper registerTrack to setTrackCount
AssetLoader declares the tracks with a setTrackCount() method. Setting
the track count on the MuxerWrapper is easier than calling
registerTrack() as many times as the number of tracks.

PiperOrigin-RevId: 496933501
2023-01-04 21:27:39 +00:00
kimvde
7c3cffdebe Change FallbackListener registerTrack to setTrackCount
AssetLoader declares the tracks with a setTrackCount() method. Setting
the track count on the FallbackListener is easier than calling
registerTrack() as many times as the number of tracks.

PiperOrigin-RevId: 496919969
2023-01-04 21:25:44 +00:00
samrobinson
59166cce9c Ensure supported request only updates changes fields.
The TransformationRequest passed to FallbackListener (and
createSupportedTransformationRequest) can have null for values that
are inferred from source. Within fallback, this can be height, width,
video mime type and audio mime type (HDR mode is not linked to source).
requestedFormat has these values populated from source, and
supportedFormat then finds the closest supported values to the
requested.

If any of the values in supportedFormat do not match the
requestedFormat, then this method would build upon the
TransformationRequest and update ALL possible fallback fields. This is
a problem because the fallback listener compares the original request
to the fallback one and notifies about all the fields that have
changed.

This CL changes this so that only the values that are not the same as
requested are changed in the supported request that is given to the
fallback listener.

PiperOrigin-RevId: 496908492
2022-12-21 15:38:48 +00:00
kimvde
cad713d0a3 Fix AS warnings and errors in FloatAudioMixingAlgorithmTest
When trying to run the test on Android Studio, error "incompatible
types: Buffer cannot be converted to ByteBuffer" is logged. This is
because ByteBuffer.flip() returns a Buffer (and not a ByteBuffer).
Annotation @CovariantReturnType on ByteBuffer.flip() should resolve this
automatically but it doesn't seem supported at the moment.
PiperOrigin-RevId: 496894723
2022-12-21 15:38:02 +00:00
kimvde
a4c3038b16 Simplify AssetLoader.Listener interface
Replace onTrackRegistered and onAllTracksRegistered with onTrackCount.

PiperOrigin-RevId: 496853037
2022-12-21 15:34:58 +00:00