18347 Commits

Author SHA1 Message Date
rohks
8a5fcf82d3 Add support to fetch ColorInfo from hvcc box in AtomParsers
#minor-release

PiperOrigin-RevId: 517086016
2023-03-16 15:34:28 +00:00
sheenachhabra
d635fa52a1 Add test for Mp4Muxer robustness
With the Mp4Muxer, while writing a large file if the something unexpected happens and muxer is not closed properly then it still outputs a valid MP4 file with partial data.

PiperOrigin-RevId: 516572804
2023-03-16 15:34:15 +00:00
Rohit Singh
49d85d625c Merge pull request #11061 from cedricxperi:dts-udts-support
PiperOrigin-RevId: 517067549
2023-03-16 15:33:38 +00:00
tofunmi
d288891c77 Clear depth buffers in glUtil.clearOutputFrame.
[GL documentation for glClear](https://registry.khronos.org/OpenGL-Refpages/gl4/html/glClear.xhtml) says "If a buffer is not present, then a glClear directed at that buffer has no effect." so it's okay to clear the depth buffer even if there isn't one set.

Also manually tested to have no impact when contrast effect and dizzy crop effect form transformer demo was added to image/video input.

PiperOrigin-RevId: 516879598
2023-03-16 15:32:45 +00:00
huangdarwin
a6479c3d10 Test: Move 4k60 test file to local from remote.
This test file used to be remote, which led to various errors in loading the file or timing out during the muxer or overall transformer.

export4k60 test timing on Pixel 7:

|Condition                 |elapsedTimeMs|AS Test "Duration" (s)|
|--------------------------|-------------|----------------------|
|local file (this CL)      |4253         |14                    |
|Remote file, googleguest  |5510         |17                    |
|Remote file, 4G           |11423        |25                    |

PiperOrigin-RevId: 516815462
2023-03-16 15:31:56 +00:00
sheenachhabra
edfef0a879 Add test for Mp4Muxer robustness
With the Mp4Muxer, while writing a large file if the something unexpected happens and muxer is not closed properly then it still outputs a valid MP4 file with partial data.

PiperOrigin-RevId: 516572804
2023-03-16 15:31:03 +00:00
michaelkatz
ea6241cc04 Extrapolate playback position during audio offload sleep
While playback thread is 'asleep' during audio offload playback, the playbackInfo.positionUs is not being constantly updated. During this time, the returned value from getCurrentPosition should return an estimate based on the most recent value and playback speed.

PiperOrigin-RevId: 516550509
2023-03-16 15:30:12 +00:00
kimvde
1ad68ffb17 Add a way in the SampleConsumer to indicate that it couldn't consume
- To support looping EditedMediaItemSequences, we need a way to tell the
AssetLoader that a sample couldn't be consumed and that it should retry
later. This is necessary in case we don't know yet whether the looping
sequence should load more samples because the other sequences haven't
made sufficient progress yet.
- The decision on whether to consume a sample is based on its timestamp
so it needs to be available.

PiperOrigin-RevId: 516546026
2023-03-16 15:29:26 +00:00
Haixia Shi
e7b1c021a5 Replace ffmpeg specific link with generic VP9/MP4 atom syntax spec. 2023-03-15 07:07:26 -07:00
Haixia Shi
614dbd1cfc Fix VP9 color space parsing in MP4 atom parser
This adds vpcC atom parsing based on the open-source FFmpeg implementation (https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/vpcc.c#L213)

This fix is needed to have correct color space parsing in vpcC atom because Instagram serves HDR VP9 encodings in MP4 containers.
2023-03-14 17:02:22 -07:00
tonihei
cebe01e87b Update Javadoc for 2.18.5
#minor-release

PiperOrigin-RevId: 516550914
2023-03-14 16:59:30 +00:00
michaelkatz
81e73f6f19 Extrapolate playback position during audio offload sleep
While playback thread is 'asleep' during audio offload playback, the playbackInfo.positionUs is not being constantly updated. During this time, the returned value from getCurrentPosition should return an estimate based on the most recent value and playback speed.

PiperOrigin-RevId: 516550509
2023-03-14 16:54:38 +00:00
kimvde
df724b517e Add a way in the SampleConsumer to indicate that it couldn't consume
- To support looping EditedMediaItemSequences, we need a way to tell the
AssetLoader that a sample couldn't be consumed and that it should retry
later. This is necessary in case we don't know yet whether the looping
sequence should load more samples because the other sequences haven't
made sufficient progress yet.
- The decision on whether to consume a sample is based on its timestamp
so it needs to be available.

PiperOrigin-RevId: 516546026
2023-03-14 16:49:47 +00:00
tonihei
dd2008b9ed Bump version numbers to Media3 1.0.0 and ExoPlayer 2.18.5
#minor-release

PiperOrigin-RevId: 516514583
(cherry picked from commit 137b40b76fe4f4d92ac02fbe56eb459693d80e2f)
2023-03-14 16:20:09 +00:00
tonihei
7f435f0c7d Update release notes for Media3 1.0.0 / ExoPlayer 2.18.5
PiperOrigin-RevId: 516478351
(cherry picked from commit 2fe2161d1bd0a31ce99a45de8b59a6358fe582fd)
2023-03-14 16:20:05 +00:00
tonihei
bfdd09b366 Update docs and READMEs for media3 stable release
- The ExoPlayer docs will link to the media3 ones (for now, we can
   permanently redirect once no further stand-alone ExoPlayer release
   is published).
 - Contibuting issues and PRs should only happen on the media3 repo.
 - The media3 README is updated to remove any pre-release status
   information.

PiperOrigin-RevId: 516234594
(cherry picked from commit bc873536f35e490a1c3d044182ecaf96dc781d3c)
2023-03-14 16:19:24 +00:00
tonihei
687362e6fe Document that custom commands can only be sent if they are defined
Custom commands from controller to session are blocked if they are
not listed in the available session command list. This isn't well
documented in the Javadoc currently.

#minor-release

PiperOrigin-RevId: 514733568
(cherry picked from commit 20669fca275554698c9176250005055b230485b3)
2023-03-14 16:19:24 +00:00
ibaker
e7f49a8cc8 Update templates to more clearly signpost between exoplayer2 and media3
Issue: google/ExoPlayer#11031

#minor-release

PiperOrigin-RevId: 514366016
(cherry picked from commit 9c82923094400e0b840a983e55be26b1dd76e7d2)
2023-03-14 16:19:24 +00:00
ibaker
14b0dcb957 Add missing @param tags
Dackka generates a warning if a method has at least one `@param` tag,
but not all of them are documented.

PiperOrigin-RevId: 513873453
(cherry picked from commit cb7d565fd47fc0b818ec0d5c7529e19df44022b5)
2023-03-14 16:19:20 +00:00
huangdarwin
f29da34dc4 Effect: Update InternalTextureManager to use double timestamps.
Instead of using a long frameDurationUs with Math.floor, use a double
frameDurationUs with Math.round,

Before, playing an image with 30 fps over 1 second would result in the final
timestamp not being aligned to the expected 1 second timestamp. Over long
periods of time, this can lead to significant timestamp drift. Additionally,
for TimestampWrapper and constrained multi-asset, where TimestampWrapper
begins the 2nd effect on the 2nd asset's startTimeUs, this floor() behavior
can lead to the first few frames of the 2nd asset using the first asset's
effects, due to timestamps being mismatched.

PiperOrigin-RevId: 516529240
2023-03-14 16:13:19 +00:00
tonihei
4fe8195c3c Bump version numbers to Media3 1.0.0 and ExoPlayer 2.18.5
#minor-release

PiperOrigin-RevId: 516514583
2023-03-14 16:09:19 +00:00
sheenachhabra
cc9b8e53da Add muxer end to end test for av1 video format
PiperOrigin-RevId: 516499906
2023-03-14 16:05:18 +00:00
samrobinson
57ce705760 Move RepeatedTranscodeTest to mh/analysis.
This test is not run on emulator, triage or postsubmit, and does not
provide value to the library in it's current state. It could be
deleted, however it still may have use for analysis of exported files
on MH.

PiperOrigin-RevId: 516491160
2023-03-14 16:01:15 +00:00
huangdarwin
9beddc2cc4 Effect: Update InternalTextureManager to use double timestamps.
Instead of using a long frameDurationUs with Math.floor, use a double
frameDurationUs with Math.round,

Before, playing an image with 30 fps over 1 second would result in the final
timestamp not being aligned to the expected 1 second timestamp. Over long
periods of time, this can lead to significant timestamp drift. Additionally,
for TimestampWrapper and constrained multi-asset, where TimestampWrapper
begins the 2nd effect on the 2nd asset's startTimeUs, this floor() behavior
can lead to the first few frames of the 2nd asset using the first asset's
effects, due to timestamps being mismatched.

PiperOrigin-RevId: 516529240
2023-03-14 15:57:32 +00:00
tonihei
8389435af1 Update release notes for Media3 1.0.0 / ExoPlayer 2.18.5
PiperOrigin-RevId: 516478351
2023-03-14 15:57:08 +00:00
tonihei
137b40b76f Bump version numbers to Media3 1.0.0 and ExoPlayer 2.18.5
#minor-release

PiperOrigin-RevId: 516514583
2023-03-14 15:56:30 +00:00
sheenachhabra
40dfe9f07d Add muxer end to end test for av1 video format
PiperOrigin-RevId: 516499906
2023-03-14 15:55:44 +00:00
samrobinson
eabf952048 Move RepeatedTranscodeTest to mh/analysis.
This test is not run on emulator, triage or postsubmit, and does not
provide value to the library in it's current state. It could be
deleted, however it still may have use for analysis of exported files
on MH.

PiperOrigin-RevId: 516491160
2023-03-14 15:54:56 +00:00
tonihei
2fe2161d1b Update release notes for Media3 1.0.0 / ExoPlayer 2.18.5
PiperOrigin-RevId: 516478351
2023-03-14 15:54:09 +00:00
Andrew Lewis
810cb6de64 Shorten labels to fix transformer demo UI 2023-03-14 10:43:18 +00:00
Andrew Lewis
4aa5467f65
Shorten labels to fix transformer demo UI 2023-03-14 10:19:30 +00:00
Andrew Lewis
e78332f9f4
Shorten labels to fix transformer demo UI 2023-03-14 10:17:10 +00:00
sheenachhabra
d579ccfd51 Remove setLocation() method from Muxer interface
PiperOrigin-RevId: 516314175
2023-03-14 09:24:06 +00:00
tofunmi
44cf57350a Update GlObjectsProvider to cover creating surface.
PiperOrigin-RevId: 516300480
2023-03-14 09:20:05 +00:00
tonihei
8a9c51a8a2 Update docs and READMEs for media3 stable release
- The ExoPlayer docs will link to the media3 ones (for now, we can
   permanently redirect once no further stand-alone ExoPlayer release
   is published).
 - Contibuting issues and PRs should only happen on the media3 repo.
 - The media3 README is updated to remove any pre-release status
   information.

PiperOrigin-RevId: 516234594
2023-03-14 09:16:04 +00:00
tianyifeng
01f04aac7f Detect audio device change from API 23 with AudioCapabilitiesReceiver
* Add AudioDeviceCallbackApi23 class which extends the AudioDeviceCallback.
* Add registerAudioDeviceCallback and unregisterAudioDeviceCallback methods for Api23.
* Modify the logics of AudioCapabilitiesReceiver constructor, register and unregister methods.

PiperOrigin-RevId: 516183997
2023-03-14 09:12:07 +00:00
christosts
ed4c3471f2 Log Format.colorInfo
Log Format.colorInfo in EventLogger and the demo app's debug text.

#minor-release

PiperOrigin-RevId: 516172936
2023-03-14 09:08:06 +00:00
bachinger
0588068f19 Support splitting the ad playback state for live streams
This change does basically three things in preparation of a follow
up CL that will insert ad breaks into multi-period live windows.

- Make sure that only `AdPlaybackState`s of multi-period streams
  are passed to `ImaUtil.splitAdPlaybackStateForPeriods()` from
  the media source.
- Enhance the splitting algorithm in ImaUtil to support live windows.
  Roughly, this includes calculating the window start position in the
  UNIX epoch context, handling the end period with an unknown duration
  and taking the `adResumePosition` of an ad group into account that
  affects the ad group start positions (`adGroup.timeUs`).
- Append a placeholder postroll ad group to the ad playback state of
  each period in a live stream to allow immediate transition to an
  ad that's inserted by an ad event from the SDK.

PiperOrigin-RevId: 516167958
2023-03-14 09:03:52 +00:00
huangdarwin
6fd6781b8d Effect: Implement TimestampWrapper.
To allow applying an effect only on a range of timestamps.

PiperOrigin-RevId: 515615662
2023-03-14 08:59:53 +00:00
samrobinson
c7350f368f Remove PCM encoding from ExportResult.
PCM encoding is a property of decoded audio, however Transformer does
not output decoded audio.

PiperOrigin-RevId: 515580886
2023-03-14 08:55:54 +00:00
kimvde
c109e396ed Add test for audio and video from different sources
PiperOrigin-RevId: 515379858
2023-03-14 08:51:52 +00:00
sheenachhabra
5acb1b310b Remove sample_h265.mp4 test file
The h265 encoding is getting covered in HDR test case (hdr10-720p.mp4) so having a separate asset is redundant here.

PiperOrigin-RevId: 515347662
2023-03-14 08:47:49 +00:00
sheenachhabra
f69160079c Add support for writing MP4 location data using media muxer
PiperOrigin-RevId: 515297752
2023-03-14 08:43:33 +00:00
tofunmi
9be419e679 Replace bitmap overlay uri EditText with local file picker
PiperOrigin-RevId: 515283537
2023-03-14 08:39:31 +00:00
sheenachhabra
84c1d851ef Add support for reading location data in MP4 extractor
The geodata is stored in the "udta" box as per MediaMuxer reference
https://cs.android.com/android/platform/superproject/+/master:frameworks/av/media/libstagefright/MPEG4Writer.cpp;drc=master;l=5588

PiperOrigin-RevId: 515095127
2023-03-14 08:35:33 +00:00
bachinger
24125f7349 Use the ad group time as content position while playing an ad
The value returned by `player.getContentPosition()` is calculated
in the timeline based on the position of the first period in the
window. In a single period live stream this position is advanced
when the live window advances on timeline refresh.

This calculation has produced slightly varying values below 1000
us which are likely caused by us/ms truncations for public API
values that we use in the IMASSAIMediaSource.

However, `AdGroup.timeUs` is the (recorded) content position at the
moment when the first ad of the an ad group has been inserted. While
playing an ad, we can always use this value instead of
`getContentPosition()` to not require recalculation.

#minor-release

PiperOrigin-RevId: 515093177
2023-03-14 08:31:16 +00:00
bachinger
37aab2b7e9 Ignore live SSAI postroll placeholders in MediaPeriodQueue
This change makes sure that the `AdPlaybackState` of any period can
contain an empty postroll placeholder.

The placeholder postroll should be represented in the `MediaPeriodId`
of a content period as `nextAdGroupIndex`, but should be ignored when
building the list of `MediaPeriodInfo` in the `MediaPeriodQueue`. This
is required to allow to add an ad group to ad playback state of the
content period that is currently being played, instantly insert an ad
period into the media period queue and immediately transition playback
to the new period.

This change makes sure and tests that

- a live server side inserted postroll placeholder can be inserted to
  a `AdPlaybackState` in well-defined and tested way (helper method)
- a postroll placeholder is NOT ignored when
  `AdPlaybackState.getAdGroupIndexAfterPositionUs` is called (this
   is required when evaluating the `nextAdGroupIndex`).
- a postroll placeholder is ignored when
  `AdPlaybackState.getAdGroupIndexForPositionUs` is called (this is
  required to not attempt to play the ad and is analogous to ignore the
  post roll placeholder in a single period timeline).
- `MediaPeriod.getFollowingMediaPeriodInfo()` does not include a
  `MediaPeriodInfo` for the placeholder postroll when building the
   queue.

PiperOrigin-RevId: 515079136
2023-03-14 08:27:16 +00:00
Googler
6514d8066d [Media3][Test] Add save location to test utils for bitmap utils
Test plan: run tests on ImageX with saving

PiperOrigin-RevId: 515068443
2023-03-14 08:23:17 +00:00
kimvde
3a11c638a5 Split TransformerEndToEndTest
Split test/ TransformerEndToEndTest into SingleMediaItemEndToEndTest and
SingleSequenceEndToEndTest to reduce the file size and split the tests
by category.

PiperOrigin-RevId: 515039502
2023-03-14 08:19:19 +00:00
kimvde
2d97d3a571 Make sure errors thrown in the PlayerListener are not swallowed
PiperOrigin-RevId: 515037497
2023-03-14 08:15:19 +00:00