9971 Commits

Author SHA1 Message Date
michaelkatz
81e73f6f19 Extrapolate playback position during audio offload sleep
While playback thread is 'asleep' during audio offload playback, the playbackInfo.positionUs is not being constantly updated. During this time, the returned value from getCurrentPosition should return an estimate based on the most recent value and playback speed.

PiperOrigin-RevId: 516550509
2023-03-14 16:54:38 +00:00
kimvde
df724b517e Add a way in the SampleConsumer to indicate that it couldn't consume
- To support looping EditedMediaItemSequences, we need a way to tell the
AssetLoader that a sample couldn't be consumed and that it should retry
later. This is necessary in case we don't know yet whether the looping
sequence should load more samples because the other sequences haven't
made sufficient progress yet.
- The decision on whether to consume a sample is based on its timestamp
so it needs to be available.

PiperOrigin-RevId: 516546026
2023-03-14 16:49:47 +00:00
huangdarwin
f29da34dc4 Effect: Update InternalTextureManager to use double timestamps.
Instead of using a long frameDurationUs with Math.floor, use a double
frameDurationUs with Math.round,

Before, playing an image with 30 fps over 1 second would result in the final
timestamp not being aligned to the expected 1 second timestamp. Over long
periods of time, this can lead to significant timestamp drift. Additionally,
for TimestampWrapper and constrained multi-asset, where TimestampWrapper
begins the 2nd effect on the 2nd asset's startTimeUs, this floor() behavior
can lead to the first few frames of the 2nd asset using the first asset's
effects, due to timestamps being mismatched.

PiperOrigin-RevId: 516529240
2023-03-14 16:13:19 +00:00
tonihei
4fe8195c3c Bump version numbers to Media3 1.0.0 and ExoPlayer 2.18.5
#minor-release

PiperOrigin-RevId: 516514583
2023-03-14 16:09:19 +00:00
samrobinson
57ce705760 Move RepeatedTranscodeTest to mh/analysis.
This test is not run on emulator, triage or postsubmit, and does not
provide value to the library in it's current state. It could be
deleted, however it still may have use for analysis of exported files
on MH.

PiperOrigin-RevId: 516491160
2023-03-14 16:01:15 +00:00
sheenachhabra
d579ccfd51 Remove setLocation() method from Muxer interface
PiperOrigin-RevId: 516314175
2023-03-14 09:24:06 +00:00
tofunmi
44cf57350a Update GlObjectsProvider to cover creating surface.
PiperOrigin-RevId: 516300480
2023-03-14 09:20:05 +00:00
tianyifeng
01f04aac7f Detect audio device change from API 23 with AudioCapabilitiesReceiver
* Add AudioDeviceCallbackApi23 class which extends the AudioDeviceCallback.
* Add registerAudioDeviceCallback and unregisterAudioDeviceCallback methods for Api23.
* Modify the logics of AudioCapabilitiesReceiver constructor, register and unregister methods.

PiperOrigin-RevId: 516183997
2023-03-14 09:12:07 +00:00
christosts
ed4c3471f2 Log Format.colorInfo
Log Format.colorInfo in EventLogger and the demo app's debug text.

#minor-release

PiperOrigin-RevId: 516172936
2023-03-14 09:08:06 +00:00
huangdarwin
6fd6781b8d Effect: Implement TimestampWrapper.
To allow applying an effect only on a range of timestamps.

PiperOrigin-RevId: 515615662
2023-03-14 08:59:53 +00:00
samrobinson
c7350f368f Remove PCM encoding from ExportResult.
PCM encoding is a property of decoded audio, however Transformer does
not output decoded audio.

PiperOrigin-RevId: 515580886
2023-03-14 08:55:54 +00:00
kimvde
c109e396ed Add test for audio and video from different sources
PiperOrigin-RevId: 515379858
2023-03-14 08:51:52 +00:00
sheenachhabra
f69160079c Add support for writing MP4 location data using media muxer
PiperOrigin-RevId: 515297752
2023-03-14 08:43:33 +00:00
sheenachhabra
84c1d851ef Add support for reading location data in MP4 extractor
The geodata is stored in the "udta" box as per MediaMuxer reference
https://cs.android.com/android/platform/superproject/+/master:frameworks/av/media/libstagefright/MPEG4Writer.cpp;drc=master;l=5588

PiperOrigin-RevId: 515095127
2023-03-14 08:35:33 +00:00
bachinger
37aab2b7e9 Ignore live SSAI postroll placeholders in MediaPeriodQueue
This change makes sure that the `AdPlaybackState` of any period can
contain an empty postroll placeholder.

The placeholder postroll should be represented in the `MediaPeriodId`
of a content period as `nextAdGroupIndex`, but should be ignored when
building the list of `MediaPeriodInfo` in the `MediaPeriodQueue`. This
is required to allow to add an ad group to ad playback state of the
content period that is currently being played, instantly insert an ad
period into the media period queue and immediately transition playback
to the new period.

This change makes sure and tests that

- a live server side inserted postroll placeholder can be inserted to
  a `AdPlaybackState` in well-defined and tested way (helper method)
- a postroll placeholder is NOT ignored when
  `AdPlaybackState.getAdGroupIndexAfterPositionUs` is called (this
   is required when evaluating the `nextAdGroupIndex`).
- a postroll placeholder is ignored when
  `AdPlaybackState.getAdGroupIndexForPositionUs` is called (this is
  required to not attempt to play the ad and is analogous to ignore the
  post roll placeholder in a single period timeline).
- `MediaPeriod.getFollowingMediaPeriodInfo()` does not include a
  `MediaPeriodInfo` for the placeholder postroll when building the
   queue.

PiperOrigin-RevId: 515079136
2023-03-14 08:27:16 +00:00
Googler
6514d8066d [Media3][Test] Add save location to test utils for bitmap utils
Test plan: run tests on ImageX with saving

PiperOrigin-RevId: 515068443
2023-03-14 08:23:17 +00:00
kimvde
3a11c638a5 Split TransformerEndToEndTest
Split test/ TransformerEndToEndTest into SingleMediaItemEndToEndTest and
SingleSequenceEndToEndTest to reduce the file size and split the tests
by category.

PiperOrigin-RevId: 515039502
2023-03-14 08:19:19 +00:00
kimvde
2d97d3a571 Make sure errors thrown in the PlayerListener are not swallowed
PiperOrigin-RevId: 515037497
2023-03-14 08:15:19 +00:00
ibaker
bfad60e661 Restrict the visibility of some DefaultDrmSession methods
These are only used from within the package, so demote them from public
to package-private.

PiperOrigin-RevId: 515029018
2023-03-14 08:07:26 +00:00
tofunmi
42aaba1f17 Update createStaticBitmapOverlay to take in context.
By making this method accept context, we can use DefaultDataSource.Factory in the DatasourceBitmapLoader to support a wider range on URI schemes in Bitmap Overlays. (and implement a local file picker for images for custom bitmap overlays in the demo transformer app)

PiperOrigin-RevId: 515013460
2023-03-14 08:03:23 +00:00
huangdarwin
82a3fd5dc0 Effect: Remove todo to consider using Gamma 2.2 everywhere.
Per testing, using gamma 2.2 everywhere resulted in undesirable color
shifting.

PiperOrigin-RevId: 515008279
2023-03-14 07:59:24 +00:00
kimvde
4b97913e2e Remove VideoSamplePipeline unused param
PiperOrigin-RevId: 515002313
2023-03-14 07:55:26 +00:00
samrobinson
73e4348411 Verify the audio properties being used within Transformer.
Add TransformerAudioEndToEndTest to emulated tests.

PiperOrigin-RevId: 514991434
2023-03-14 07:51:18 +00:00
tianyifeng
bb9ac21434 Handle dead object error during writing buffer to AudioTrack
PiperOrigin-RevId: 514989776
2023-03-14 07:47:23 +00:00
kimvde
87d1c3e624 Fix threading issueis in SequenceAssetLoaderListener
Callbacks onTrackCount and onTrackAdded can be called simultaneously
from different threads.

Before this fix, it was possible for the MuxerWrapper and
FallbackListener track count to never be set, or to be set
with incorrect values.

PiperOrigin-RevId: 514779719
2023-03-14 07:43:16 +00:00
tonihei
fe710871aa Use current position code when checking if AudioTrack has pending data
We currently rely on the raw playback head position to check if any
data is pending in the AudioTrack (e.g. to know if the renderer is
still ready). We can use the value returned from getCurrentPositionUs
instead to align the "isReady" logic with the playback position logic.

This has the side effect that getPlaybackHeadPosition position is called
less often when the position is obtained via getTimestamp.

PiperOrigin-RevId: 514747613
2023-03-14 07:39:10 +00:00
tofunmi
82ab327e02 Update effect to take in and use a GlObjectsProvider
PiperOrigin-RevId: 514744747
2023-03-14 07:34:53 +00:00
rohks
0bec22f31b Fix copybara import in GlTextureInfo
PiperOrigin-RevId: 514728583
2023-03-07 15:48:06 +00:00
tonihei
0113174788 Make DefaultDashChunkSource more robust against empty segment timelines
Segment timelines are technically allowed to be empty, but not all places
add the necessary checks.

Issue: google/ExoPlayer#11014
PiperOrigin-RevId: 514722205
2023-03-07 15:43:41 +00:00
rohks
d83b4a358f Fix copybara import
PiperOrigin-RevId: 514720434
2023-03-07 15:06:27 +00:00
samrobinson
83192e870c Fix AudioMixer javadoc.
PiperOrigin-RevId: 514699994
2023-03-07 15:02:21 +00:00
huangdarwin
3b58a74d0c GL: Reduce visibility of GlProgram.loadAsset.
This is only used within the same class, so can be private.

PiperOrigin-RevId: 514665000
2023-03-07 12:57:18 +00:00
samrobinson
2d22604efe Pass Metadata to Muxer when adding a track.
PiperOrigin-RevId: 514575400
2023-03-07 12:53:00 +00:00
tofunmi
e12de4cc1f Create GlObjectsProvider
To create this file TextureInfo has been moved to common and renamed to GLTextureInfo.

We'll look to expand the interface in future to cover more of the methods around GL object maintenance in future as required.

PiperOrigin-RevId: 514445397
2023-03-07 12:48:40 +00:00
kimvde
ce9fa88c0d Rename CompositeAssetLoader to SequenceAssetLoader
This clarifies that a SequenceAssetLoader loads data corresponding to an
EditedMediaItemSequence.

PiperOrigin-RevId: 514442681
2023-03-07 12:44:22 +00:00
tofunmi
f94d3cf6c4 ImageAssetLoaderTest minor fixes
PiperOrigin-RevId: 514436359
2023-03-07 12:40:08 +00:00
tofunmi
6188952682 Add ImageAssetLoader tests.
PiperOrigin-RevId: 514431184
2023-03-07 12:35:55 +00:00
kimvde
55f2d1a5e2 Make onOutputFormat nullable
- This is to make sure we know about all the tracks before initializing
the SamplePipelines. This allows to set the muxer and the fallback
listener track count before the SamplePipelines are built.
- As a result, the test files had to be updated because the order in
which the tracks are written has changed.
- The ImageAssetLoader also had to be updated to call onOutputFormat
repeatedly until it returns a non-null SampleConsumer.
- Also fix the trackCount sent to the muxer and fallback listener. The
correct track count can be computed now that we know about all the
tracks before building the SamplePipelines.

PiperOrigin-RevId: 514426123
2023-03-07 12:31:43 +00:00
huangdarwin
67e359c867 GL: Delete frame buffers after use.
Before, we used to never call glDeleteFramebuffers, which could
in theory lead to leaks in the number of frame buffers
available and make releasing the GL context more expensive.

PiperOrigin-RevId: 514387847
2023-03-07 12:27:32 +00:00
sheenachhabra
db6064288c Create orientation matrix byte array using util method
PiperOrigin-RevId: 514375835
2023-03-07 12:23:14 +00:00
ibaker
d67c869378 Add missing @param tags
Dackka generates a warning if a method has at least one `@param` tag,
but not all of them are documented.

#minor-release

PiperOrigin-RevId: 513873453
2023-03-07 12:14:41 +00:00
ibaker
af45bedfcd Ensure ForwardingPlayer users do listener registration correctly
The `@CallSuper` annotation should help catch cases where subclasses are
calling `delegate.addListener` instead of `super.addListener` but it
will also (unintentionally) prevent subclasses from either completely
no-opping the listener registration, or implementing it themselves in a
very custom way. I think that's probably OK, since these cases are
probably unusual, and they should be able to suppress the warning/error.

Issue: androidx/media#258

#minor-release

PiperOrigin-RevId: 513848402
2023-03-07 12:06:07 +00:00
tofunmi
e8c86e1bae Remove effect/SimpleBitmapLoader & replace with DataSourceBitmapLoader
PiperOrigin-RevId: 513824487
2023-03-07 12:01:50 +00:00
kimvde
db540a02b7 Make sure that tracks are transcoded/transmuxed consistently
All audio tracks should either all be transcoded or all be transmuxed.
Same for video tracks.

To achieve this, simplify the behaviour of transmuxAudio/Video.

PiperOrigin-RevId: 513809287
2023-03-07 11:57:35 +00:00
samrobinson
93b9cc731d Provide TransformerInternal with the AssetLoader output format.
TI can use this Format as part of creating the SamplePipelines.

PiperOrigin-RevId: 513777622
2023-03-07 11:53:03 +00:00
tonihei
af6807d959 Fix lint-baseline.xml for latest UI translations
#minor-release

PiperOrigin-RevId: 513533248
2023-03-02 17:39:57 +00:00
andrewlewis
9f0a6f1619 Fix typo
PiperOrigin-RevId: 513529059
2023-03-02 17:35:30 +00:00
andrewlewis
389d5ea229 Fix name for FrameCacheGlShaderProgram
This implements `GlShaderProgram` (and is GL-specific).

PiperOrigin-RevId: 513528160
2023-03-02 17:31:16 +00:00
samrobinson
8642594379 Add getNormalizedTrackType, mapping image to video track type.
PiperOrigin-RevId: 513514142
2023-03-02 17:22:29 +00:00
tonihei
3b16231e29 Bump version numbers to Media3 1.0.0-rc02 and ExoPlayer 2.18.4
#minor-release

PiperOrigin-RevId: 513488487
2023-03-02 13:28:55 +00:00