10333 Commits

Author SHA1 Message Date
Rohit Singh
d88dd74b2b Merge pull request #11064 from haixia-meta:release-v2
PiperOrigin-RevId: 517128752
2023-03-16 16:36:19 +00:00
samrobinson
3ac82b1d62 Add test cases for mixing to mono.
PiperOrigin-RevId: 517127021
2023-03-16 16:27:41 +00:00
rohks
e4446c37fb Add support to fetch ColorInfo from hvcc box in AtomParsers
#minor-release

PiperOrigin-RevId: 517086016
2023-03-16 16:10:16 +00:00
Rohit Singh
eb6fc93b15 Merge pull request #11061 from cedricxperi:dts-udts-support
PiperOrigin-RevId: 517067549
2023-03-16 16:01:28 +00:00
tofunmi
ddbad57031 Clear depth buffers in glUtil.clearOutputFrame.
[GL documentation for glClear](https://registry.khronos.org/OpenGL-Refpages/gl4/html/glClear.xhtml) says "If a buffer is not present, then a glClear directed at that buffer has no effect." so it's okay to clear the depth buffer even if there isn't one set.

Also manually tested to have no impact when contrast effect and dizzy crop effect form transformer demo was added to image/video input.

PiperOrigin-RevId: 516879598
2023-03-16 15:52:28 +00:00
huangdarwin
e6343a34b8 Test: Move 4k60 test file to local from remote.
This test file used to be remote, which led to various errors in loading the file or timing out during the muxer or overall transformer.

export4k60 test timing on Pixel 7:

|Condition                 |elapsedTimeMs|AS Test "Duration" (s)|
|--------------------------|-------------|----------------------|
|local file (this CL)      |4253         |14                    |
|Remote file, googleguest  |5510         |17                    |
|Remote file, 4G           |11423        |25                    |

PiperOrigin-RevId: 516815462
2023-03-16 15:43:35 +00:00
Haixia Shi
e7b1c021a5 Replace ffmpeg specific link with generic VP9/MP4 atom syntax spec. 2023-03-15 07:07:26 -07:00
Haixia Shi
614dbd1cfc Fix VP9 color space parsing in MP4 atom parser
This adds vpcC atom parsing based on the open-source FFmpeg implementation (https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/vpcc.c#L213)

This fix is needed to have correct color space parsing in vpcC atom because Instagram serves HDR VP9 encodings in MP4 containers.
2023-03-14 17:02:22 -07:00
michaelkatz
81e73f6f19 Extrapolate playback position during audio offload sleep
While playback thread is 'asleep' during audio offload playback, the playbackInfo.positionUs is not being constantly updated. During this time, the returned value from getCurrentPosition should return an estimate based on the most recent value and playback speed.

PiperOrigin-RevId: 516550509
2023-03-14 16:54:38 +00:00
kimvde
df724b517e Add a way in the SampleConsumer to indicate that it couldn't consume
- To support looping EditedMediaItemSequences, we need a way to tell the
AssetLoader that a sample couldn't be consumed and that it should retry
later. This is necessary in case we don't know yet whether the looping
sequence should load more samples because the other sequences haven't
made sufficient progress yet.
- The decision on whether to consume a sample is based on its timestamp
so it needs to be available.

PiperOrigin-RevId: 516546026
2023-03-14 16:49:47 +00:00
huangdarwin
f29da34dc4 Effect: Update InternalTextureManager to use double timestamps.
Instead of using a long frameDurationUs with Math.floor, use a double
frameDurationUs with Math.round,

Before, playing an image with 30 fps over 1 second would result in the final
timestamp not being aligned to the expected 1 second timestamp. Over long
periods of time, this can lead to significant timestamp drift. Additionally,
for TimestampWrapper and constrained multi-asset, where TimestampWrapper
begins the 2nd effect on the 2nd asset's startTimeUs, this floor() behavior
can lead to the first few frames of the 2nd asset using the first asset's
effects, due to timestamps being mismatched.

PiperOrigin-RevId: 516529240
2023-03-14 16:13:19 +00:00
tonihei
4fe8195c3c Bump version numbers to Media3 1.0.0 and ExoPlayer 2.18.5
#minor-release

PiperOrigin-RevId: 516514583
2023-03-14 16:09:19 +00:00
samrobinson
57ce705760 Move RepeatedTranscodeTest to mh/analysis.
This test is not run on emulator, triage or postsubmit, and does not
provide value to the library in it's current state. It could be
deleted, however it still may have use for analysis of exported files
on MH.

PiperOrigin-RevId: 516491160
2023-03-14 16:01:15 +00:00
sheenachhabra
d579ccfd51 Remove setLocation() method from Muxer interface
PiperOrigin-RevId: 516314175
2023-03-14 09:24:06 +00:00
tofunmi
44cf57350a Update GlObjectsProvider to cover creating surface.
PiperOrigin-RevId: 516300480
2023-03-14 09:20:05 +00:00
tianyifeng
01f04aac7f Detect audio device change from API 23 with AudioCapabilitiesReceiver
* Add AudioDeviceCallbackApi23 class which extends the AudioDeviceCallback.
* Add registerAudioDeviceCallback and unregisterAudioDeviceCallback methods for Api23.
* Modify the logics of AudioCapabilitiesReceiver constructor, register and unregister methods.

PiperOrigin-RevId: 516183997
2023-03-14 09:12:07 +00:00
christosts
ed4c3471f2 Log Format.colorInfo
Log Format.colorInfo in EventLogger and the demo app's debug text.

#minor-release

PiperOrigin-RevId: 516172936
2023-03-14 09:08:06 +00:00
huangdarwin
6fd6781b8d Effect: Implement TimestampWrapper.
To allow applying an effect only on a range of timestamps.

PiperOrigin-RevId: 515615662
2023-03-14 08:59:53 +00:00
samrobinson
c7350f368f Remove PCM encoding from ExportResult.
PCM encoding is a property of decoded audio, however Transformer does
not output decoded audio.

PiperOrigin-RevId: 515580886
2023-03-14 08:55:54 +00:00
kimvde
c109e396ed Add test for audio and video from different sources
PiperOrigin-RevId: 515379858
2023-03-14 08:51:52 +00:00
sheenachhabra
f69160079c Add support for writing MP4 location data using media muxer
PiperOrigin-RevId: 515297752
2023-03-14 08:43:33 +00:00
sheenachhabra
84c1d851ef Add support for reading location data in MP4 extractor
The geodata is stored in the "udta" box as per MediaMuxer reference
https://cs.android.com/android/platform/superproject/+/master:frameworks/av/media/libstagefright/MPEG4Writer.cpp;drc=master;l=5588

PiperOrigin-RevId: 515095127
2023-03-14 08:35:33 +00:00
bachinger
37aab2b7e9 Ignore live SSAI postroll placeholders in MediaPeriodQueue
This change makes sure that the `AdPlaybackState` of any period can
contain an empty postroll placeholder.

The placeholder postroll should be represented in the `MediaPeriodId`
of a content period as `nextAdGroupIndex`, but should be ignored when
building the list of `MediaPeriodInfo` in the `MediaPeriodQueue`. This
is required to allow to add an ad group to ad playback state of the
content period that is currently being played, instantly insert an ad
period into the media period queue and immediately transition playback
to the new period.

This change makes sure and tests that

- a live server side inserted postroll placeholder can be inserted to
  a `AdPlaybackState` in well-defined and tested way (helper method)
- a postroll placeholder is NOT ignored when
  `AdPlaybackState.getAdGroupIndexAfterPositionUs` is called (this
   is required when evaluating the `nextAdGroupIndex`).
- a postroll placeholder is ignored when
  `AdPlaybackState.getAdGroupIndexForPositionUs` is called (this is
  required to not attempt to play the ad and is analogous to ignore the
  post roll placeholder in a single period timeline).
- `MediaPeriod.getFollowingMediaPeriodInfo()` does not include a
  `MediaPeriodInfo` for the placeholder postroll when building the
   queue.

PiperOrigin-RevId: 515079136
2023-03-14 08:27:16 +00:00
Googler
6514d8066d [Media3][Test] Add save location to test utils for bitmap utils
Test plan: run tests on ImageX with saving

PiperOrigin-RevId: 515068443
2023-03-14 08:23:17 +00:00
kimvde
3a11c638a5 Split TransformerEndToEndTest
Split test/ TransformerEndToEndTest into SingleMediaItemEndToEndTest and
SingleSequenceEndToEndTest to reduce the file size and split the tests
by category.

PiperOrigin-RevId: 515039502
2023-03-14 08:19:19 +00:00
kimvde
2d97d3a571 Make sure errors thrown in the PlayerListener are not swallowed
PiperOrigin-RevId: 515037497
2023-03-14 08:15:19 +00:00
Cedric T
3909db2a4e Add DTSUHDSpecificBox (udts) support to parseAudioSampleEntry(). 2023-03-14 16:12:45 +08:00
ibaker
bfad60e661 Restrict the visibility of some DefaultDrmSession methods
These are only used from within the package, so demote them from public
to package-private.

PiperOrigin-RevId: 515029018
2023-03-14 08:07:26 +00:00
tofunmi
42aaba1f17 Update createStaticBitmapOverlay to take in context.
By making this method accept context, we can use DefaultDataSource.Factory in the DatasourceBitmapLoader to support a wider range on URI schemes in Bitmap Overlays. (and implement a local file picker for images for custom bitmap overlays in the demo transformer app)

PiperOrigin-RevId: 515013460
2023-03-14 08:03:23 +00:00
huangdarwin
82a3fd5dc0 Effect: Remove todo to consider using Gamma 2.2 everywhere.
Per testing, using gamma 2.2 everywhere resulted in undesirable color
shifting.

PiperOrigin-RevId: 515008279
2023-03-14 07:59:24 +00:00
kimvde
4b97913e2e Remove VideoSamplePipeline unused param
PiperOrigin-RevId: 515002313
2023-03-14 07:55:26 +00:00
samrobinson
73e4348411 Verify the audio properties being used within Transformer.
Add TransformerAudioEndToEndTest to emulated tests.

PiperOrigin-RevId: 514991434
2023-03-14 07:51:18 +00:00
tianyifeng
bb9ac21434 Handle dead object error during writing buffer to AudioTrack
PiperOrigin-RevId: 514989776
2023-03-14 07:47:23 +00:00
kimvde
87d1c3e624 Fix threading issueis in SequenceAssetLoaderListener
Callbacks onTrackCount and onTrackAdded can be called simultaneously
from different threads.

Before this fix, it was possible for the MuxerWrapper and
FallbackListener track count to never be set, or to be set
with incorrect values.

PiperOrigin-RevId: 514779719
2023-03-14 07:43:16 +00:00
tonihei
fe710871aa Use current position code when checking if AudioTrack has pending data
We currently rely on the raw playback head position to check if any
data is pending in the AudioTrack (e.g. to know if the renderer is
still ready). We can use the value returned from getCurrentPositionUs
instead to align the "isReady" logic with the playback position logic.

This has the side effect that getPlaybackHeadPosition position is called
less often when the position is obtained via getTimestamp.

PiperOrigin-RevId: 514747613
2023-03-14 07:39:10 +00:00
tofunmi
82ab327e02 Update effect to take in and use a GlObjectsProvider
PiperOrigin-RevId: 514744747
2023-03-14 07:34:53 +00:00
rohks
0bec22f31b Fix copybara import in GlTextureInfo
PiperOrigin-RevId: 514728583
2023-03-07 15:48:06 +00:00
tonihei
0113174788 Make DefaultDashChunkSource more robust against empty segment timelines
Segment timelines are technically allowed to be empty, but not all places
add the necessary checks.

Issue: google/ExoPlayer#11014
PiperOrigin-RevId: 514722205
2023-03-07 15:43:41 +00:00
rohks
d83b4a358f Fix copybara import
PiperOrigin-RevId: 514720434
2023-03-07 15:06:27 +00:00
samrobinson
83192e870c Fix AudioMixer javadoc.
PiperOrigin-RevId: 514699994
2023-03-07 15:02:21 +00:00
huangdarwin
3b58a74d0c GL: Reduce visibility of GlProgram.loadAsset.
This is only used within the same class, so can be private.

PiperOrigin-RevId: 514665000
2023-03-07 12:57:18 +00:00
samrobinson
2d22604efe Pass Metadata to Muxer when adding a track.
PiperOrigin-RevId: 514575400
2023-03-07 12:53:00 +00:00
tofunmi
e12de4cc1f Create GlObjectsProvider
To create this file TextureInfo has been moved to common and renamed to GLTextureInfo.

We'll look to expand the interface in future to cover more of the methods around GL object maintenance in future as required.

PiperOrigin-RevId: 514445397
2023-03-07 12:48:40 +00:00
kimvde
ce9fa88c0d Rename CompositeAssetLoader to SequenceAssetLoader
This clarifies that a SequenceAssetLoader loads data corresponding to an
EditedMediaItemSequence.

PiperOrigin-RevId: 514442681
2023-03-07 12:44:22 +00:00
tofunmi
f94d3cf6c4 ImageAssetLoaderTest minor fixes
PiperOrigin-RevId: 514436359
2023-03-07 12:40:08 +00:00
tofunmi
6188952682 Add ImageAssetLoader tests.
PiperOrigin-RevId: 514431184
2023-03-07 12:35:55 +00:00
kimvde
55f2d1a5e2 Make onOutputFormat nullable
- This is to make sure we know about all the tracks before initializing
the SamplePipelines. This allows to set the muxer and the fallback
listener track count before the SamplePipelines are built.
- As a result, the test files had to be updated because the order in
which the tracks are written has changed.
- The ImageAssetLoader also had to be updated to call onOutputFormat
repeatedly until it returns a non-null SampleConsumer.
- Also fix the trackCount sent to the muxer and fallback listener. The
correct track count can be computed now that we know about all the
tracks before building the SamplePipelines.

PiperOrigin-RevId: 514426123
2023-03-07 12:31:43 +00:00
huangdarwin
67e359c867 GL: Delete frame buffers after use.
Before, we used to never call glDeleteFramebuffers, which could
in theory lead to leaks in the number of frame buffers
available and make releasing the GL context more expensive.

PiperOrigin-RevId: 514387847
2023-03-07 12:27:32 +00:00
sheenachhabra
db6064288c Create orientation matrix byte array using util method
PiperOrigin-RevId: 514375835
2023-03-07 12:23:14 +00:00
ibaker
d67c869378 Add missing @param tags
Dackka generates a warning if a method has at least one `@param` tag,
but not all of them are documented.

#minor-release

PiperOrigin-RevId: 513873453
2023-03-07 12:14:41 +00:00