8786 Commits

Author SHA1 Message Date
huangdarwin
371c5c1b2e Transformer GL: Split out ScaleToFit and Advanced GlFrameProcessors
* Move auto-adjustments for transformation matrices from the
  VideoTranscodingSamplePipeline constructor to the new
  ScaleToFitFrameProcessor.
* Add GlFrameProcessor#getOutputDimensions() to allow for GlFrameProcessors with
  different input and output dimensions. This is a prerequisite for
  Presentation.
* Tested with unit tests (and manually just in case).
* A follow up CL will implement change the FrameProcessor input to be scale and
  rotate values as requested by the user. This was kept out of this CL to
  reduce CL review size. Presentation will also be implemented in a follow up
  CL.

PiperOrigin-RevId: 434774854
2022-03-15 16:36:52 +00:00
huangdarwin
8763843c84 Transformer: Add javadoc to VideoTranscodingSamplePipeline.
As pointed out in a previous review, createFallbackTransformationRequest
can be a bit confusing to parse.

Added javadocs and renamed parameters appropriately, to make it slightly
more easy to understand.

PiperOrigin-RevId: 434733313
2022-03-15 13:24:28 +00:00
olly
dacf8f847c Rollback of ff6e641fbb
*** Original commit ***

Rollback of 20282151b5

*** Original commit ***

Fix PlayerView touch handling

Overriding onTouchEvent was causing multiple issues, and
appears to be unnecessary. Removing the override fixes:

1. StyledPlayerView accessibility issue where "hide player
   controls" actually toggled play/pause.
2. Delivery of events to a registered OnClick...

***

PiperOrigin-RevId: 434502423
2022-03-15 00:52:19 +00:00
hschlueter
255007623d Clarify GlProgram parameter name.
PiperOrigin-RevId: 434441008
2022-03-15 00:45:50 +00:00
ibaker
06639e0a2f Fix stale javadoc in DashUtil
The type of this parameter was changed when it was moved from
OfflineLicenseHelper to DashUtil in
<unknown commit>

PiperOrigin-RevId: 433770325
2022-03-15 00:42:29 +00:00
hschlueter
64d174b39c Move inputTexId parameter to GlFrameProcessor#initialize().
This parameter will not change between frames in the near
future.

PiperOrigin-RevId: 433765986
2022-03-15 00:38:59 +00:00
claincly
da2f61e035 Fix namings in SsimHelper.
There's no use of encoders in SsimHelper, changed all to "decoder".

PiperOrigin-RevId: 433730292
2022-03-15 00:32:30 +00:00
olly
22159673bd Move TrackSelection back to ExoPlayer module
PiperOrigin-RevId: 433729648
2022-03-15 00:29:14 +00:00
ibaker
018d177012 Rollback of 9ec8678525
*** Original commit ***

Don't call MediaDrm.setLogSessionId in FrameworkMediaDrm

This method throws an UnsupportedOperationException on some Android 12
devices.

***

PiperOrigin-RevId: 433708582
2022-03-15 00:25:49 +00:00
huangdarwin
b31cd08d7e Transformer GL: Rename TexCoords to TexSampleCoords.
The variable marks the coordinates used to sample from a texture, so hopefully
this makes the naming a bit more descriptive.

This renames vTexCoords and aTexCoords. No functional changes intended.

PiperOrigin-RevId: 433499934
2022-03-09 17:48:18 +00:00
bachinger
e044235421 Transform map of AdsLoader.State to a Bundle
This fixes an exception thrown when parceling becasue the type can not be found
(expects the AdsPlaybackState to be Serializable). Transforming the map and the
ad playback states to a Bundle fixes the problem.

#minor-release

PiperOrigin-RevId: 433491993
2022-03-09 17:40:43 +00:00
huangdarwin
a703223534 Transformer GL: Remove unused setResolution exception javadoc.
PiperOrigin-RevId: 433467719
2022-03-09 14:55:33 +00:00
ibaker
153b16a888 Version bump to exoplayer:2.17.1 and media3:1.0.0-alpha03
#minor-release

PiperOrigin-RevId: 433467068
2022-03-09 14:51:19 +00:00
bachinger
5bf123165f Avoid rebuffering at the end of SSAI post-rolls
When the start position of a MediaPeriodInfo is equal or higher than the duration,
we set the start position to `duration - 1` to end on the last frame. With server
side inserted ad streams, this has the effect that we actually need to seek back to
the last content frame after a post-roll.

This is desirable when actually ending on that frame but produces a BUFFERING event
when transitioning from an SSAI stream with a post-roll to the next media item in
the playlist. This change sets the start position to the duration when we are
clipping the last content period of an SSAI stream that is played in a playlist.

PiperOrigin-RevId: 433445680
2022-03-09 12:22:02 +00:00
hschlueter
f5796b2d48 Use background thread for FrameEditor's OpenGL calls.
If an OpenGL call blocks because the encoder's input surface is full,
this will now block the background thread while the main thread can
continue querying encoder output and free up encoder capacity until
it accepts more input unblocking the background thread.

PiperOrigin-RevId: 433283287
2022-03-09 15:37:30 +00:00
olly
ff6e641fbb Rollback of 20282151b5
*** Original commit ***

Fix PlayerView touch handling

Overriding onTouchEvent was causing multiple issues, and
appears to be unnecessary. Removing the override fixes:

1. StyledPlayerView accessibility issue where "hide player
   controls" actually toggled play/pause.
2. Delivery of events to a registered OnClickListener when
   useController is false.
3. Delivery of events to a registered OnLongClickListener
   in all configurations.
4. Incorrectly treating a sequence of touch events that
   exit the bounds of the vi...

***

PiperOrigin-RevId: 433262414
2022-03-09 15:34:01 +00:00
samrobinson
82a50154db Disable calculating SSIM on instrumentation tests.
PiperOrigin-RevId: 433237266
2022-03-09 15:30:18 +00:00
claincly
f40f97fa79 Improve resolution fallback logic.
With the new version, we try the following before fixing resolution:

- Fix size alignment
- Try 3/4 the width and height
- Try 2/3 the width and height
- Try 1/2 the width and height

Also: align the resolution ends in 1 or 9 to 0.
PiperOrigin-RevId: 433206358
2022-03-09 15:26:54 +00:00
samrobinson
0770d55c1a Nit cleanup of SsimHelper.
PiperOrigin-RevId: 433174767
2022-03-09 15:23:20 +00:00
olly
20282151b5 Fix PlayerView touch handling
Overriding onTouchEvent was causing multiple issues, and
appears to be unnecessary. Removing the override fixes:

1. StyledPlayerView accessibility issue where "hide player
   controls" actually toggled play/pause.
2. Delivery of events to a registered OnClickListener when
   useController is false.
3. Delivery of events to a registered OnLongClickListener
   in all configurations.
4. Incorrectly treating a sequence of touch events that
   exit the bounds of the view before ACTION_UP as a click,
   both for delivery to OnClickListener and for toggling
   the controls.

Note: After this change, control visibility will not be
toggled if the application developer explicitly sets the
view to be non-clickable. I think that's probably working
as intended though. It seems correct that a non-clickable
view would not respond to clicks.

Issue: google/ExoPlayer#8627
Issue: google/ExoPlayer#9605
Issue: google/ExoPlayer#9861
PiperOrigin-RevId: 433016626
2022-03-09 15:19:43 +00:00
olly
b2a5298e27 Decouple UI module from ExoPlayer
This change rewrites the UI module's track selection
components to depend on the Player API, allowing us to
finally remove the UI module's dependency on ExoPlayer
as a concrete player implementation.

PiperOrigin-RevId: 432989318
2022-03-09 15:16:18 +00:00
claincly
661d7ddedb Record transformation duration in TransformerAndroidTestRunner.
The change will be useful in testing transcoding performance

PiperOrigin-RevId: 432956283
2022-03-09 15:12:53 +00:00
samrobinson
3877171f18 Remove SSIM calculation on audio only output.
PiperOrigin-RevId: 432937645
2022-03-09 15:09:27 +00:00
samrobinson
8adc145fa2 Add regression test forcing encode/decode.
PiperOrigin-RevId: 432928418
2022-03-09 15:05:29 +00:00
claincly
e5c229be00 Use getSupportedWidth/HeightFor() API for finding resolution.
Previously, we've used getSupportedHeights/Widths() to find the supported
resolution. However, the height/width can be over-reported when using these
APIs. For example, getSupportedWidths and getSupportedHeights can both return
3840, but the supported height when using 3840 as width is only 2160.

PiperOrigin-RevId: 432926192
2022-03-09 15:02:07 +00:00
bachinger
25e5680415 Ignore MetadataRenderer when evaluating SSAI period transitions
This makes the reading period advance early as expected at the end of an ad
period. Before this change the reading position of the metadata renderer
prevented advancing the period until metadata arrived after the start position of
the following period. Only then the reading position of the metadata renderer
is updated and beyond the start position of the following period which is a
condition to advance the reading period.

Because transitioning to the next period is a virtual transition and the
SharedMediaPeriod keeps reading from the same underlying sample streams, the
metadata renderer can safely be ignored for this check.

#minor-release

PiperOrigin-RevId: 432646037
2022-03-05 14:16:43 +00:00
olly
d7df426069 Simplify application of track overrides
PiperOrigin-RevId: 432485797
2022-03-04 18:54:25 +00:00
olly
aafe5f48bd DownloadHelper: Add getTracksInfo
We will be migrating our track selection UI components to be
based on TracksInfo. We need DownloadHelper to expose TracksInfo
to make it compatible with such components.

PiperOrigin-RevId: 432474487
2022-03-04 18:10:45 +00:00
claincly
3f615040c0 Add method to disable passthrough.
PiperOrigin-RevId: 432461547
2022-03-04 17:15:16 +00:00
olly
b83c2cb8cb DownloadHelper: Support multiple track selection overrides
addTrackSelectionForSingleRenderer takes a list of legacy overrides,
which are then set on the supplied parameters one at a time to run
track selection. This allows multiple overrides for a single track
type to be applied in the download use case, despite it not being
possible to place such overrides directly into a single parameters.

For new style overrides, multiple overrides for the same track type
can be placed directly into a single parameters. Therefore we'll be
able to replace use of addTrackSelectionForSingleRenderer with use
of addTrackSelection, which is a much cleaner API. For this to work,
we need to make DownloadHelper apply multiple overrides in this case.

PiperOrigin-RevId: 432459834
2022-03-04 17:06:26 +00:00
samrobinson
039de75f1f Move AndroidTestUtil run methods to a TransformerAndroidTestRunner.
This will allow for easier customisation of the additional tasks
performed by the test runner, such as calculating metrics like SSIM.

PiperOrigin-RevId: 432434850
2022-03-04 14:52:55 +00:00
ibaker
52a8caedf6 Remove unused constants from (Styled/Legacy)PlayerView
These were added in
b57aa34b66
and then not removed when the rest of the code was removed in
0b0277af50

PiperOrigin-RevId: 432419684
2022-03-04 13:07:27 +00:00
ibaker
41276e8b9b Remove media3 PlayerView javadoc references to overriding layouts
These should have been removed as part of 1391b7c65d, since we no
longer officially support overriding the layout file for this class.

This class is known as StyledPlayerView in exoplayer2.

#minor-release

PiperOrigin-RevId: 432411322
2022-03-07 11:38:51 +00:00
andrewlewis
e66d0c9039 Fix E-AC3 output capability check without sample rate
#minor-release

PiperOrigin-RevId: 432189509
2022-03-07 11:35:09 +00:00
ibaker
540f2061cb Deprecate SingleSampleMediaSource.Factory#setTrackId
This method is no longer needed since we added SubtitleConfiguration#id
in 59d98b9a4e.

Issue: google/ExoPlayer#10016

#minor-release

PiperOrigin-RevId: 432169262
2022-03-07 11:31:27 +00:00
olly
2cb2f6ded4 DownloadHelper: Accept generic TrackSelectionParameters
DownloadHelper is in the ExoPlayer module, so there's no reason
why it can't use ExoPlayer specific track selections. That said,
we want our UI components to operate on generic
TrackSelectionParameters, and we want such UI components to be
useful for selecting tracks for download. To keep this interop,
it's necessary to have DownloadHelper accept generic
TrackSelectionParameters, or to require application code to
convert them. The first approach seems preferable!

PiperOrigin-RevId: 432158846
2022-03-03 12:34:30 +00:00
olly
ad81d5dd20 TrackSelectionOverride: Remove select-all-tracks constructor
This constructor always does the wrong thing for non-adaptive groups
containing more than 1 track, because it'll incorrectly generate an
adaptive selection. Replace it with a constructor for specifying a
single track within the group instead.

PiperOrigin-RevId: 431673458
2022-03-03 12:27:28 +00:00
hschlueter
e53dbf893e Use asset with increasing timestamps for matrix transformation test.
PiperOrigin-RevId: 431658068
2022-03-03 12:23:53 +00:00
hschlueter
550299db9a Avoid duplicate logging of GL exceptions.
Only log GL exceptions in GlUtil if they aren't thrown. Otherwise,
it's up to the caller whether or not to log them to avoid logging them
twice.

PiperOrigin-RevId: 431657397
2022-03-03 12:20:18 +00:00
hschlueter
1fe19195f9 Don't delete textures before destroying OpenGL context.
Destroying the context destroys all of OpenGL (see
https://www.khronos.org/opengl/wiki/OpenGL_Context), so deleting
textures is redundant.

PiperOrigin-RevId: 431653728
2022-03-03 12:16:48 +00:00
hschlueter
15f9abdd74 Add an instrumentation unit test for TransformationFrameProcessor.
This test tests the same cases as the FrameEditorDataProcessingTest
as currently the main FrameEditor functionality is to apply a
transformation matrix using a TransformationFrameProcessor.

PiperOrigin-RevId: 431642066
2022-03-03 12:13:12 +00:00
samrobinson
084bde2d91 Restructure regression androidTests, adding a 4k60 video and test.
Test list can be structured so tests don't need to be in different
classes, cleaning up the location of new test cases.

PiperOrigin-RevId: 431492941
2022-03-01 10:24:54 +00:00
claincly
85512f66e6 Add SSIM support to AndroidTestUtil.
PiperOrigin-RevId: 431479473
2022-03-01 10:21:22 +00:00
hschlueter
651efecf75 Use Color helper for packing in BitmapTestUtil.
PiperOrigin-RevId: 431451974
2022-03-01 10:18:02 +00:00
olly
b4e05e00f2 TrackSelectionParameters: Simplify disabling of track types
As evidenced by the somewhat awkward logic in PlayerControlView, the
previous design wasn't very friendly to expected usage. There will be
more usage when the track selection dialog components are migrated,
which would be similarly awkward without this change.

PiperOrigin-RevId: 431407675
2022-03-01 10:14:38 +00:00
Ian Baker
f66c9290ad Merge pull request #10011 from tonykwok:dev-v2
PiperOrigin-RevId: 431395359
2022-03-01 10:11:06 +00:00
claincly
85ce0bc181 Add SSIM helper for transcoding quality measurements.
We use SSIM to measure the transcoding quality between. SSIM is a widely used
tool that compares the luma channel between two images, and generates a score
from 0 to 1 that indicates "how similar" the two images are.

In `SsimHelper`, we decode the two videos, extract matching frames and
calculates the mean SSIM (SSIM averaged all matching frames) for both videos.
Matching frames are referred to as "comparisonFrame" in the CL, which is
selected based on the frame number and a user-set comparison interval.
For instance, if the interval is 7, then every seventh frames are compared.

We use MediaCodec/MediaExtractor to decode the video, and use ImageReader to
extract the decoded frame.

The SSIM calculation logic is a inspired by and modified from the CTS
[MSSIMMatcher](https://cs.android.com/android/platform/superproject/+/master:cts/tests/tests/uirendering/src/android/uirendering/cts/bitmapcomparers/MSSIMComparer.java;l=1?q=mssimcom)
that has some errors and extra features we don't need (like handling RGB
images).

Adds TranscodeQualityTest to ensure high quality transcoding.

PiperOrigin-RevId: 430951206
2022-03-01 10:00:32 +00:00
samrobinson
41e1b11c75 Allow the difference Bitmap to be saved to device cache.
When investigating how 'bad' a failure is, it's useful to see the diff
between input and output bitmaps.

PiperOrigin-RevId: 430917732
2022-03-01 09:56:45 +00:00
ibaker
bbfeb27600 Rollback of 9961d918a9
*** Original commit ***

Rollback of 8d9c4f4774

*** Original commit ***

Rollback of 43b796b64d

*** Original commit ***

Rollback of 08c5b1cb0a

*** Original commit ***

PiperOrigin-RevId: 430905772
2022-03-01 09:49:59 +00:00
olly
1bc4ba29f5 FMP4: Fix output of mixed v0 and v1 emsg samples
Issue: google/ExoPlayer#9996
#minor-release
PiperOrigin-RevId: 430773329
2022-03-01 09:46:31 +00:00