9723 Commits

Author SHA1 Message Date
kimvde
a043c8c204 Avoid re-encoding if video effects are no-op
This is to avoid regressions introduced by removing the convenience
methods from TransformationRequest.

PiperOrigin-RevId: 502864512
2023-01-23 13:08:38 +00:00
tonihei
50f066d634 Correctly filter PlayerInfo by available getter commands.
When bundling PlayerInfo, we need to remove information if the
controller is not allowed to access it. This was only partially
done at the moment.

PiperOrigin-RevId: 502852798
2023-01-23 13:04:16 +00:00
samrobinson
a44032c59b Add ColorInfo to TransformationResult.
TESTED: Manually verified the ColorInfo is propagated when VSP is used.
PiperOrigin-RevId: 502843773
2023-01-23 12:59:59 +00:00
rohks
8fcd6bbffc Fix javadoc references to writeSampleData
PiperOrigin-RevId: 502821506
2023-01-18 13:26:52 +00:00
samrobinson
76af4270a9 Generate silent audio if no audio track is available.
To always generate silent audio, `removeAudio(true)` can be used in conjunction.

PiperOrigin-RevId: 502814315
2023-01-18 13:21:57 +00:00
kimvde
6b6e4993b9 Add EditedMediaItem class
This class will be used in a follow-up CL.

PiperOrigin-RevId: 502615323
2023-01-18 13:16:48 +00:00
tonihei
5461d5cbf1 Make availableCommands known when bundling PlayerInfo
When bundling PlayerInfo, we remove data when the controller is not
allowed to access this data via getters. We also remove data for
performance reasons. In the toBundle() method, it's currently hard to
make the connection between allowed commands and filtering, because
the values are checked at a different place. This can be made more
readable by forwarding the applicable Commands directly.

The only functional fix is to filter the Timeline when sending the
first PlayerInfo after a connecting a controller if the command to
get the Timeline is not available. This also allows us to remove a
path to filter MediaItems from Timelines as it isn't used.

PiperOrigin-RevId: 502607391
2023-01-18 13:11:49 +00:00
samrobinson
2cc64a037c Add resolution fallback for 25% of requested.
If a device only supports 1920x1080 as maximum resolution, then adding
the 25% case ensures 8k is reduced correctly.

PiperOrigin-RevId: 502588364
2023-01-18 13:06:39 +00:00
samrobinson
5da086fe01 Add a unit test to check 2/3 resolution supported.
PiperOrigin-RevId: 502573469
2023-01-18 13:01:35 +00:00
Googler
345f2345c7 Disables play/pause button when there's nothing to play
PiperOrigin-RevId: 502571320
2023-01-18 12:56:41 +00:00
kimvde
55aeb76e0c Add CompositeAssetLoader
This is an AssetLoader that wraps a sequence of AssetLoaders. It will
be used for constrained multi-asset.

This class can currently only concatenate media items with the exact
same format.

PiperOrigin-RevId: 502525796
2023-01-18 12:51:50 +00:00
kimvde
89e14989b3 Remove setScale and setRotationDegrees from TransformationRequest
Also remove usages of TransformationRequest convenience methods
(setScale, setRotationDegrees and setResolution).

Some usages of setResolution can't be removed yet because they are used
for fallback.

PiperOrigin-RevId: 502415748
2023-01-17 03:38:19 +00:00
sheenachhabra
2656284e63 Align muxer code with media3 coding conventions.
This CL includes following changes:
1. Remove GCA related terms/links from java docs and comments.
2. Make class final where ever possible.
3. Append /* package */ for default classes.
4. Change java docs to recommended format.
5. Replace term "packet" with "sample" to avoid confusion.
6. Correct TODO format.
7. Delete MediaFormatUtil.java from muxer module and add its methods into MediaFormatUtil.java in common module.

Note: The java doc on various boxes has the limited description which was already present. In future I am planning to add proper small description for each box (from MP4 spec).

Not included in this CL:
1. Order of element correction as it will show lot of changes and might create confusion with other minor changes.
2. Correction in test cases (Only some renaming).

PiperOrigin-RevId: 502414139
2023-01-17 03:34:21 +00:00
kimvde
a4f9f9487b Add the possility to shift frame timestamps in SampleConsumer
This is needed for constrained multi-asset to shift the timestamps of
the media items that are not the first in the sequence.

PiperOrigin-RevId: 502409923
2023-01-17 03:30:32 +00:00
huangdarwin
e28d434c9e Effect: Automatically save bitmaps in pixel test.
Also, omit the "actual" label from output files, as this boilerplate isn't necessary
(it doesn't disambiguate between any other saved filename like "expected").

PiperOrigin-RevId: 502378188
2023-01-17 03:22:36 +00:00
huangdarwin
9789b39c78 HDR: Use human-readable values for ColorInfo.toString().
This should make debugging much easier as values will be more human-readable.
Before this CL, one needed to reference MediaFormatUtil to check the
colorSpace/colorTransfer/colorRange values and make sure values were as
expected.

PiperOrigin-RevId: 502367147
2023-01-17 03:18:35 +00:00
huangdarwin
27d44e86fe Effect: Create GlEffectsFrameProcessorTestRunner.
Makes GlEffectsFrameProcessorPixelTest slightly more modular in preparation for
copying this into transformer/mh.

Refactoring change. No functional change intended.

PiperOrigin-RevId: 501902223
2023-01-17 03:14:24 +00:00
samrobinson
eb49666049 Move SonicAudioProcessor to media3.common module.
PiperOrigin-RevId: 501881646
2023-01-17 03:10:32 +00:00
claincly
42aecce353 Add a frame cache
Frame cache compensates for the fluctuation in frame processing times.

Imagine a frame takes 10ms to process, and the interval between two frames is
33ms. The third frame took 40ms to process.

If we don't have frame cache:
- Process frame 1, ready after 10ms, starts playback, now t=0 ms
- Start processing frame 2, ready at t=10ms,
- Release frame 2 at t=33ms
- We start processing the third frame at t=33ms
- The third frame is due presentation at t=66ms
- But frame 3 is available at t=73ms, late

If we have a frame cache of say 3 frams,
- Process frame 1, ready after 10ms, starts playback, now t=0 ms
- Start processing frame 2, ready at t=10ms
- Start processing frame 3, ready at t=50ms
- Release frame 2 at t=33ms
- Start frame 4, ready at t=60ms
- Frame 3 is due presentation at t=66ms
- Frame 3 isn't late

PiperOrigin-RevId: 501869948
2023-01-17 03:06:36 +00:00
claincly
c86d70a98b Allow forcing duration in FrameworkMuxer.
PiperOrigin-RevId: 501865706
2023-01-17 03:02:43 +00:00
claincly
07fe60710c Fix not able to set a null output surface.
Previously, after calling MCVR.setOutput() with null, `frameProcessorManager`'s output surface is cleared. What was unexpected is `ExoPlayerInternal` notifies a zero output resolution after clearing the output surface. This zero resolution causes FrameProcessor to fail.

PiperOrigin-RevId: 501861993
2023-01-17 02:58:45 +00:00
huangdarwin
e773031927 Effect: Move BitmapTestUtil to common test util class.
Move BitmapTestUtil from media3.effect to media3.test.utils.

This allows this class to be accessed from both transformer and effect tests.

Refactoring change. No functional change intended.

PiperOrigin-RevId: 501837130
2023-01-17 02:54:54 +00:00
huangdarwin
1a1cdcc350 HDR: Update glsl color transfer errors to use unique colors.
Each error will make the output be a solid color with a unique color.

PiperOrigin-RevId: 501824355
2023-01-17 02:50:57 +00:00
huangdarwin
4ade37c48d HDR: Implement PQ to SDR tone-mapping.
Tested manually on the Pixel 7 and Samsung S10.

PiperOrigin-RevId: 501626354
2023-01-17 02:46:59 +00:00
claincly
91d43dc915 Handle FrameProcessor output size change
Notifies event listener of FrameProcessor's new output size to layout the
PlayerView

PiperOrigin-RevId: 501590239
2023-01-17 02:43:03 +00:00
huangdarwin
4f4f28b923 Transformer: Clarify setResolution javadoc.
This class may be removed soon, but in the meantime clarify this method's
javadoc summary fragment to make it marginally clearer that this only affects
the "displayed" height instead of the "encoded" height.

PiperOrigin-RevId: 501582219
2023-01-17 02:39:07 +00:00
samrobinson
298ad765a3 Make ToInt16AudioProcessor available in common.audio.
PiperOrigin-RevId: 501575932
2023-01-17 02:35:14 +00:00
huangdarwin
27e71d2f91 Transformer: Simplify EncoderWrapper rotationDegrees logic.
Small refactoring change to simplify the logic a tiny bit by removing one
unnecessary variable. No functional change intended.

PiperOrigin-RevId: 501566533
2023-01-17 02:31:20 +00:00
samrobinson
8ccb8b64ae Move BaseAudioProcessor abstract class to common.
This is a prerequisite step to making the `ResamplingAudioProcessor` public.

PiperOrigin-RevId: 501545561
2023-01-17 02:27:27 +00:00
bachinger
055ed77433 Document that DownloadService needs notification permissions
Starting with Android 13 (API 33) an app needs to request the
permission to post notifications or notifications are suppressed.
This change documents this in the class level JavaDoc of the
`DownloadService`.

Issue: google/ExoPlayer#10884
PiperOrigin-RevId: 501346908
2023-01-17 02:19:33 +00:00
samrobinson
8fe90562a6 Allow Muxer.writeSampleData to take @C.BufferFlag int flags.
PiperOrigin-RevId: 501314812
2023-01-17 02:11:33 +00:00
claincly
74cc588049 Fix typo in ExoPlayer.java
PiperOrigin-RevId: 501287961
2023-01-17 02:07:28 +00:00
tofunmi
a786c03a34 Move BitmapLoader to common.
The migration strategy is to deprecate `androidx.media3.session.BitmapLoader` and copy the file into common since BitmapLoader is a public interface that apps could be relying on.

PiperOrigin-RevId: 501266521
2023-01-17 02:03:35 +00:00
kimvde
bbffa8ae27 Remove setDecoderFactory from Transformer
It makes more sense to have it in the AssetLoader now that decoding is
done there, particularly as the decoder factory should rarely be
customized.

Also remove the setter from the AssetLoader.Factory because it's
uncommon to have setters in a factory.

PiperOrigin-RevId: 501244974
2023-01-17 01:59:37 +00:00
kimvde
a5288a20fb Add a queue at the start of the buffer SamplePipelines
This improves performance and makes the code more intuitive.

PiperOrigin-RevId: 501220234
2023-01-17 01:55:45 +00:00
kimvde
4a91a234a8 Remove setters from AssetLoader.Factory
It's unusual to have setters on a Factory.

PiperOrigin-RevId: 501212264
2023-01-17 01:51:45 +00:00
steveanton
25fdc4ed47 Add an audio mixer that combines concurrent audio sources
Design document: go/me-android-audio-mixer

PiperOrigin-RevId: 501048758
2023-01-10 21:03:03 +00:00
kimvde
3a0fa0c89b Indicate that some FrameProcessor methods can be called on any thread
The FrameProcessor is created on the GL thread, but:
- setInputFrameInfo() is currently called from the playback thread.
- release() is currently called from the transformer internal thread.

PiperOrigin-RevId: 501035719
2023-01-10 20:59:10 +00:00
tonihei
09a15fb731 Update bandwidth meter estimates
PiperOrigin-RevId: 501010994
2023-01-10 20:55:15 +00:00
tonihei
3c6b8c5d7b Handle AV sync timestamps when draining the audio sink during tunneling
When audio processors are enabled during tunneling, they must produce
output immediately, ensuring that the timestamps of the output samples
correspond to the input and that no additional samples are produced.
This requirement is documented in the Javadoc of DefaultAudioSink.

However, this alone doesn't guarantee all buffers are immediately
written to the AudioTrack, because the AudioTrack writes are
non-blocking and may need multiple attempts.

When draining the audio sink at the end of the stream, we currently
fail in this situation because we assert that the timestamp must be
set (=the drain operation is a no-op). But this may not be true when
the previous non-blocking write wasn't fully handled. We can fix this
by saving the last timestamp and reusing it during draining.

Issue: google/ExoPlayer#10847
PiperOrigin-RevId: 500943891
2023-01-10 20:51:19 +00:00
samrobinson
2693b6025f Make EncoderBitrateProvider and implementations package private.
There is no API to allow applications to use these currently. Until
that exists, these classes do not need to be public.

PiperOrigin-RevId: 500792401
2023-01-10 20:47:24 +00:00
samrobinson
8cac27a77c Add audio and video values from Format to TransformationResult.
PiperOrigin-RevId: 500789076
2023-01-10 20:43:34 +00:00
samrobinson
8ad26c0d8b Split IAE and ISE cases in MuxerWrapper and improve javadoc.
PiperOrigin-RevId: 500786191
2023-01-10 20:39:43 +00:00
samrobinson
b3831778ef Improve docs and split audio/video order in TransformationResult.
PiperOrigin-RevId: 500776260
2023-01-10 20:35:52 +00:00
kimvde
886063d649 Rename SampleConsumer dequeueInputBuffer to getInputBuffer
This makes more sense as SampleConsumer.dequeueInputBuffer returns the
same buffer as long as it is not queued.

PiperOrigin-RevId: 500631982
2023-01-10 20:31:59 +00:00
kimvde
c642f811a6 Fix sample interleaving
Whether to write a sample or not was based on the timestamp of the
previous sample, rather than the current sample.

PiperOrigin-RevId: 500195279
2023-01-10 20:28:01 +00:00
kimvde
ee37b453dd Split ExoPlayerAssetLoaderRenderer
Split ExoPlayerAssetLoaderRenderer into audio and video renderers.

PiperOrigin-RevId: 500102256
2023-01-10 20:24:03 +00:00
rohks
d49437c6e1 Initialise fields used for bundling as String directly
Initialising the fields as Integer and then getting a String on compute time is slow. Instead we directly initialise these fields as String. Improves the time taken in bundling PlayerInfo further to less than 200ms from ~300ms.

Also modified a test to improve productive coverage.

PiperOrigin-RevId: 500003935
2023-01-10 20:20:08 +00:00
rohks
bdd6818c14 Fix typo in DefaultTrackSelector.Parameters field
PiperOrigin-RevId: 499905136
2023-01-10 20:16:03 +00:00
samrobinson
03bb91401d Refactor MuxerWrapper handling of track details.
This brings together the multiple details about a muxer track, and
reduces the need for additional variables and more complicated track
tracking.

PiperOrigin-RevId: 499872145
2023-01-10 20:12:05 +00:00