17541 Commits

Author SHA1 Message Date
kimvde
07ff48165a Handle when some MediaItems require transcoding but others don't
- For single-asset, the behavior stays the same. Transcode if and only
  if it's necessary,
- For constrained multi-asset, always transcode, except if the setter to
  transmux is set. This is to avoid failing if a MediaItem that doesn't
  require transcoding is followed by a MediaItem that does require
  transcoding.

PiperOrigin-RevId: 508097798
2023-02-14 16:02:45 +00:00
Googler
287cc3a570 Fix tests affected by Robolectric's constructor visibility bug
Previously, Robolectric's instrumentation updated all constructors to be
public. This caused two main types of problems:

1) When non-public constructors were made public and added to the Android API,
Robolectric allowed tests to incorrectly use the constructors on older SDK
levels (where they were non-public). This most commonly occurs for
AccessibiltyEvent and AccessibilityNodeInfo.

2) When reflection was used to instantiate classes that were instrumented by
Robolectric, all constructors were accessible.

Fix issues across Google3 Robolectric tests that were affected by this issue.
A forthcoming change will fix the instrumentation in Robolectric to prevent
this type of issue from occurring.

Tested:
    TAP --sample ran all affected tests and none failed
    http://test/OCL:507861075:BASE:507805409:1675803313108:f2128fa4
PiperOrigin-RevId: 508087822
2023-02-14 15:58:45 +00:00
sheenachhabra
1a0b75e763 Move AudioBoxesTest.java to third_party/.../muxer/...
PiperOrigin-RevId: 508062795
2023-02-14 15:54:38 +00:00
michaelkatz
2590dd5ef5 Encapsulate Opus frames in Ogg during audio offload
PiperOrigin-RevId: 508053559
2023-02-08 15:54:41 +00:00
kimvde
45c42ed373 Add an API entry point to pass a Composition
PiperOrigin-RevId: 508031337
2023-02-08 15:50:43 +00:00
tonihei
79b688ef30 Advance position across transition for readahead renderer error
If a renderer error happens while processing readahead data for the
next item in the playlist, we currently throw the error immediately
and only set the item id in the error details. This makes it harder
to associate the error to the right item. For example, the user
facing UI is likely not updated to show the failing item when the
error is reported.

This can be improved slighly by force setting the position to the
failing item. The playback still fails immediately, but this can't
be avoided because the renderer itself went into an error state.

PiperOrigin-RevId: 507808635
2023-02-08 15:42:50 +00:00
microkatz
08342ea9da Merge pull request #248 from lemondoglol:update-segment-size
PiperOrigin-RevId: 507784608
2023-02-08 15:38:48 +00:00
huangdarwin
01082a48f1 Effect: Create GlEffect.isNoOp to simplify Transformer.
This will also allow us to skip transcodes in the future for other effects, like
cropping, color effects, etc.

PiperOrigin-RevId: 507765618
2023-02-08 15:34:42 +00:00
kimvde
f39ae692c8 Rename TransformationResult to ExportResult
Also replace some usages of deprecated Transformer listeners with the new
ones.

PiperOrigin-RevId: 507743860
2023-02-08 15:30:40 +00:00
tonihei
4ede931c2a Fix AudioTrackPositionTracker logic for playback speed adjustments
The AudioTrackPositionTracker needs to correct positions by
the speed set on the AudioTrack itself whenever it makes
estimations based on real-time (=the real-time playout
duration is not equal to the media duration played).

This happens for the main playback path already, but not for
the mode in which the position is estimated from the playback
head position and also not in the phase after the track has
been stopped. Both cases are not very noticeable during
normal playback, but become relevant when playing in offload
mode.

PiperOrigin-RevId: 507736408
2023-02-08 15:26:39 +00:00
claincly
14881f0ef0 Remove debug print.
Checked there's no other occurrence.

PiperOrigin-RevId: 507534909
2023-02-08 15:22:37 +00:00
huangdarwin
659b107022 Effect: Rename TextureProcessor to ShaderProgram.
Also, replace instances of "texture processor" to "shader program", with capitalization retained.

PiperOrigin-RevId: 507515655
2023-02-08 15:18:38 +00:00
tofunmi
af259054af Update CompositeAssetLoader to override queueInputBitmap
PiperOrigin-RevId: 507515587
2023-02-08 15:14:39 +00:00
christosts
80dbaca0e5 Detect HEVC HDR10 codec profile more accurately
In MediaCodecUtil, use Format.colorInfo, besides the codec string,
to accurately map to a 10bit HEVC profile.

PiperOrigin-RevId: 507500071
2023-02-08 15:10:37 +00:00
claincly
274617de17 Flush FrameProcessor when appropriate.
PiperOrigin-RevId: 507461550
2023-02-08 15:06:34 +00:00
bachinger
91e611d5dc Create an AdEventListener for each supported stream type
This is a refactoring to separate and simplify the logic
of VOD and live streams when handling IMA ad events. An
additional listener will be required for DASH live stream
in a follow-up CL.

PiperOrigin-RevId: 507435741
2023-02-08 15:02:36 +00:00
kimvde
4631105fb3 Rename startTransformation to start
Also update some Javadoc in Transformer

PiperOrigin-RevId: 507395956
2023-02-08 14:58:38 +00:00
tofunmi
c32794f0cc Update sample pipelines and frame processors to handle image input.
PiperOrigin-RevId: 506965394
2023-02-08 14:54:39 +00:00
huangdarwin
7684949fa4 HDR: Move codec errors from VideoSamplePipeline
It doesn't actually make sense for them to be placed in the Transformer, because the error's root causes are actually only in codecs. Also, a few codec errors were
repeated, so deduplicate these instances

PiperOrigin-RevId: 506937695
2023-02-08 14:50:44 +00:00
tofunmi
d4db00e51a Update FrameProcessor.create() to accept an input track type.
Based on [this conversation thread](https://chat.google.com/room/AAAA--f88ao/76Rem_cRCK8), I've opted to update the existing FrameProcessor.create() rather than deprecate it, as it is unlikely to be in use by apps outside google3.

PiperOrigin-RevId: 506920930
2023-02-08 14:46:49 +00:00
huangdarwin
557b7dcc71 Demo: Request READ_MEDIA_VIDEO on API 33+
More info in: https://developer.android.com/about/versions/13/behavior-changes-13#granular-media-permissions

PiperOrigin-RevId: 506908079
2023-02-08 14:42:52 +00:00
kimvde
87ff3157c9 Update Composition structure to take List instead of ImmutableList
This is more flexible for apps and more consitent with the Player API.

PiperOrigin-RevId: 506901645
2023-02-08 14:38:57 +00:00
samrobinson
73d4b67e46 Expand createNoSupportedMimeTypeException for HDR ColorInfo case.
PiperOrigin-RevId: 506900817
2023-02-08 14:35:02 +00:00
kimvde
90de454e1e Adapt TransformationResult for multi-asset
PiperOrigin-RevId: 506898392
2023-02-08 14:31:02 +00:00
samrobinson
937fcf9c4a Refactor TransformationException.createForCodec method overloads.
* Overload added `(cause, errorCode, isVideo, isDecoder, details)`,
where `details` is a string of values to be added to the error message
of the `TransformationException`.
* Overload with `MediaFormat` and `mediaCodecName` moved to
`DefaultCodec`, because all usages of that overload were from
`DefaultCodec`, and this allows a simplified API because of internally
stored values.
* `mediaCodecName` removed from overload that takes a `Format`.
* Reordered `createForCodec` parameters.

PiperOrigin-RevId: 506895268
2023-02-08 14:27:04 +00:00
kimvde
83074c0dcd Rename Transformer callbacks
PiperOrigin-RevId: 506890459
2023-02-08 14:23:02 +00:00
samrobinson
eb4b6f812c Implement a ChannelMixingAudioProcessor.
PiperOrigin-RevId: 506886903
2023-02-08 14:19:05 +00:00
claincly
c09904e143 Prefer hardware encoders in EncoderSelector.
In encoding small and odd-numbered resolutions, like `316x61` ([this image](https://upload.wikimedia.org/wikipedia/commons/6/65/100winners.png)), the current fallback logic prefers a software encoder to hardware ones. The assumption was, the encoder factory applies the encoder size alignment and changes the resolution to `316x60` for SW encoders and `316x64` for HW ones. SW encoders is selected because the supported resolution 60 is closer to requested 61, than the hardware supported 64.

This change changes the default encoder selection process to only expose hardware encoders if there is any.

PiperOrigin-RevId: 506879983
2023-02-08 14:15:03 +00:00
kimvde
14210d3df3 Rename inputFormat to firstInputFormat in multiple places
With multi-asset, the sample pipelines can process more than one
MediaItem. The renaming makes it clear that the format passed to the
SamplePipeline constructors is the one corresponding to the first
MediaItem. Indeed, the first format is the one used to configure the
SamplePipelines.

PiperOrigin-RevId: 506879260
2023-02-08 14:11:01 +00:00
samrobinson
bfee268e3f Turn off ExoPlayer metrics reporting when used by Transformer.
This avoids affecting ExoPlayer metrics with non-typical usage.

PiperOrigin-RevId: 506878231
2023-02-08 14:06:59 +00:00
tofunmi
8a8fd8dbe5 Update duration & frame rate javadoc in EditedMediaItem
PiperOrigin-RevId: 506874799
2023-02-08 14:03:02 +00:00
kimvde
3635fe7abe Add MediaItem change listener
PiperOrigin-RevId: 506868341
2023-02-08 13:59:03 +00:00
kimvde
23a544025d Use CompositeAssetLoader in Transformer
PiperOrigin-RevId: 506863538
2023-02-08 13:55:01 +00:00
huangdarwin
4528ea3005 HDR: Fix debug surface by setting correct output color.
Set the correct output color on the debug SurfaceViewWrapper, so that SDR contents
can have an output transfer of either GAMMA_2_2 (Gamma 2.2) or SDR (SMPTE 170M).

This fixes an issue where in-app tone-mapping would output gamma 2.2, and the
SDR value incorrectly hardcoded here would lead to an error in the OpenGL, which
does not support SMPTE 170M.

PiperOrigin-RevId: 506684602
2023-02-08 13:50:58 +00:00
tofunmi
8121709605 Add ImageAssetLoader and ImageConfiguration.
PiperOrigin-RevId: 506619637
2023-02-08 13:46:57 +00:00
bachinger
7269d2a4c2 Publish ConcatenatingMediaSource2
Can be used to combine multiple media items into a single timeline window.

Issue: androidx/media#247
Issue: google/ExoPlayer#4868
PiperOrigin-RevId: 506283307
(cherry picked from commit eb8fffba15810b8206653d8ffaff233d823fbfc9)
2023-02-02 16:47:22 +00:00
christosts
76e87406bb Merge pull request #10793 from fraunhoferfokus:dash-thumbnail-support
PiperOrigin-RevId: 506261584
(cherry picked from commit 107e0c6e42c6fd080b1be19e5305597cf880a9dd)
2023-02-02 16:47:22 +00:00
claincly
976e3c72d1 Fix end of stream handling for previewing.
Before this CL, the `renderedLastFrame` flag is not set if the last frame is released immediately (force render), or when it's dropped.

PiperOrigin-RevId: 506358626
2023-02-02 16:13:36 +00:00
claincly
77cc25242b Report flushing completed after all pending frames are decoded.
With the current ExtTexMgr,

it can happen that

- `x` frames are registered, but haven't arrived yet
- flush
  - need to drop `x` frames when they arrive on SurfaceTexture
  - status is reset to 0 pending, 0 available, drop `x` when frames arrive
- register one frame
  - status is set to 1 pending, 0 available, drop `x` when frames arrive
- flush
  - now the number of frame to drop is reset to `pending - available = 1`
  - but it should be `x+1`

This CL solves the issue by reporting (by running the afterFlushTask) flush completes only after all the pending frames before calling flush are accounted for.

PiperOrigin-RevId: 506310671
2023-02-02 16:09:29 +00:00
samrobinson
83bc474f3c Clarify case where HDR encoder and muxer have no shared mime type.
The prior version (with the call to createEncodingException) could
never occur as select...SupportedMimeType already checks for HDR
editing support. This change ensures we throw before creating an
encoder, gives a better error code and allows future simplifications
around createForCodec (see child CL).

PiperOrigin-RevId: 506308290
2023-02-02 16:05:21 +00:00
kimvde
b8ec982ee3 Pass a Composition to Transformer callbacks
PiperOrigin-RevId: 506304101
2023-02-02 16:01:06 +00:00
claincly
b2ab4393a4 Support flushing in FrameProcessor
Flushing resets all the texture processors within the `FrameProcessor`. This
includes:

- At the back, the FinalMatrixTextureProcessorWrapper, and its MatrixTextureProcessor
- At the front, the ExternalTextureManager
- All the texture processors in between
- All the ChainingGlTextureProcessorListeners in between texture processors
- All the internal states in the aforementioned components

The flush process follows the order, from `GlEffectsFrameProcessor.flush()`

1. Flush the `FrameProcessingTaskExecutor`, so that after it returns, all tasks queued before calling `flush()` completes
2. Post to `FrameProcessingTaskExecutor`, to flush the `FinalMatrixTextureProcessorWrapper`
3. Flushing the `FinalMatrixTextureProcessorWrapper` will propagate flushing through, via the `ChainingGlTextureProcessorListener`

Startblock:
   has LGTM from christosts
   and then
   add reviewer andrewlewis
PiperOrigin-RevId: 506296469
2023-02-02 15:56:55 +00:00
tofunmi
eea1868d54 Add sRGB eotf to overlay fragment shader.
The eotf is needed so that overlay (image) colors are correctly interpreted and mixed the linear video colors.

Also replaces the 100winners.png with "homemade" image file.

Added GlEffectsFrameProcessor test to justify that the color looks correct at the end of frame processing.

PiperOrigin-RevId: 506290309
2023-02-02 15:52:46 +00:00
ibaker
4dfa7ca249 Fix (another) LeanbackPlayerAdapter param name mismatch
I missed this when fixing `positionInMs` for Dackka in d2a3d8f6fa

This time I manually verified that all the `@Override` methods have
parameter names that match [the docs](https://developer.android.com/reference/androidx/leanback/media/PlayerAdapter).

#minor-release

PiperOrigin-RevId: 506017063
(cherry picked from commit 736f090cce540de78a968e62fd8c5aec413e8122)
2023-02-02 13:18:47 +00:00
tonihei
284bf97007 Fix timestamp comparison for seeks in fMP4
When seeking in fMP4, we try to extract as little samples as possible
by only starting at the preceding sync frame. This comparison should
use <= to allow sync frames at exactly the seek position.

Issue: google/ExoPlayer#10941

#minor-release

PiperOrigin-RevId: 505098172
(cherry picked from commit ac3017b5805cb98b4292e4bbe1f7181f47684b88)
2023-02-02 13:18:47 +00:00
michaelkatz
ace97facf2 Match MergingMediaPeriod track selection by period index in id
MergingMediaPeriod creates its track groups with ids concatenating position in its periods array and the underlying child track group id. The ids can be used in selectTracks for matching to periods list.

Issue: google/ExoPlayer#10930
PiperOrigin-RevId: 505074653
(cherry picked from commit ee055ef004686ddb3844666f9506a95c6e16a59f)
2023-02-02 13:18:46 +00:00
tonihei
b9bb3235c2 Tweak UI behavior when commands are missing.
For most missing commands, we already disable the corresponding
controls. This change extends this to more UI elements that are
disabled in case the corresponding action is unavailable.

#minor-release

PiperOrigin-RevId: 505057751
(cherry picked from commit 641c3b1b22cc67133151b7af9905473485b0f51c)
2023-02-02 13:18:46 +00:00
Balachandar KM
3c09cbd8d4
Handle SQLiteException when trying to accessing DatabaseProvider#getReadableDatabase() 2023-02-02 10:54:06 +00:00
bachinger
eb8fffba15 Publish ConcatenatingMediaSource2
Can be used to combine multiple media items into a single timeline window.

Issue: androidx/media#247
Issue: google/ExoPlayer#4868
PiperOrigin-RevId: 506283307
2023-02-01 15:36:40 +00:00
christosts
107e0c6e42 Merge pull request #10793 from fraunhoferfokus:dash-thumbnail-support
PiperOrigin-RevId: 506261584
2023-02-01 15:32:27 +00:00