9288 Commits

Author SHA1 Message Date
huangdarwin
e6a1936ba9 Effect: Add some FrameProcessor javadoc.
In particular, make it a bit more clear that "rendering" and "releasing" frames are
related concepts, and how they differ from one another in conjunction with frame
dropping.

PiperOrigin-RevId: 471037733
2022-10-19 20:21:43 +00:00
Googler
674b3d4575 Fix 19 ErrorProneStyle findings:
* Non-standard parameter comment; prefer `/* paramName= */ arg`
  (see http://go/bugpattern/ParameterComment) (19 times)

This CL looks good? Just LGTM and Approve it!
This CL doesn’t look good? This is what you can do:
* Revert this CL, by replying "REVERT: <provide reason>"
* File a bug under go/error-prone-bug for category ErrorProneStyle if there's an issue with the CL content.
* File a bug under go/rosie-bug if there's an issue with how the CL was managed.
* Revert this CL and not get a CL that cleans up these paths in the future by
replying "BLOCKLIST: <provide reason>". This is not reversible! We recommend to
opt out the respective paths in your CL Robot configuration instead:
go/clrobot-opt-out.

This CL was generated by CL Robot - a tool that cleans up code findings
(go/clrobot). The affected code paths have been enabled for CL Robot in //depot/google3/java/com/google/android/libraries/media/METADATA which is reachable following include_presubmits from //depot/google3/third_party/java_src/android_libs/media/METADATA.
Anything wrong with the signup? File a bug at go/clrobot-bug.

#codehealth

Tested:
    Local presubmit tests passed.
PiperOrigin-RevId: 471022923
2022-10-19 20:14:12 +00:00
ibaker
e285e70785 Remove media3-only line from exoplayer2 build.gradle file
#minor-release

PiperOrigin-RevId: 470999044
2022-10-19 20:03:06 +00:00
andrewlewis
1023254d8e Update color info mismatch test
This should now expect transformation to succeed.

PiperOrigin-RevId: 470950411
2022-10-19 19:55:44 +00:00
andrewlewis
f1a3a40eb1 Log instead of throwing for transfer mismatch
PiperOrigin-RevId: 470354448
2022-10-19 19:52:00 +00:00
rohks
142d1c062c Switch incorrectly configured native multidex to legacy
Native multidex can only be used for binaries with minSdkVersion of 21 or higher, but minSdkVersion was specified to 16.

PiperOrigin-RevId: 470003836
2022-10-19 19:39:59 +00:00
huangdarwin
a2139109e3 HDR: Add PQ support.
Use the PQ OETF and EOTF to ensure that intermediate fragment shader operations
using PQ are in linear BT.2020 rather than PQ and HLG-1 BT.2020.

Also, swap the OETF and EOTF in shaders, as they were used incorrectly before

Manually tested by verifying transformer demo HLG and PQ videos look the same with and without this CL, including with a BitmapOverlayProcessor enabled to test flows both with one MatrixTransformationProcessor that skips HDR TFs, and with one that doesn't.

PiperOrigin-RevId: 469736067
2022-10-19 19:24:54 +00:00
claincly
5cf166c157 Fix outdated RtpDataLoadable javadoc
PiperOrigin-RevId: 469443086
2022-10-19 19:21:09 +00:00
claincly
c611435521 Handle initial RTSP seek
PiperOrigin-RevId: 469143613
2022-10-19 19:09:50 +00:00
Googler
010ecec89d Reassign TODO to new bug.
PiperOrigin-RevId: 468672505
2022-10-19 19:06:04 +00:00
Googler
4e7f9c575d Change onOutputFrameAvailable timestamp from nanos to micros.
Upstream timestamps from the decoder are also in microseconds,
so using microseconds here is consistent with that.

PiperOrigin-RevId: 468659099
2022-10-19 18:57:51 +00:00
Googler
2c063546a1 Allow frame release to be controlled outside FrameProcessor.
Adds a method to FrameProcessor.Listener to be called when an
output frame is available and a method releaseOutputFrame in
FrameProcessor allowing the caller to trigger release of the
oldest available output frame at a given timestamp. Late frames
or frames with unset release times are dropped in the
FinalMatrixTransformationProcessorWrapper.

More than one output frame can become available before they are
released if the penultimate GlTextureProcessor is capable of producing
multiple output frames. Processing continues while waiting for
releaseOutputFrame to be called. Frame release tasks are prioritized
over other tasks.

PiperOrigin-RevId: 468473072
2022-10-19 18:50:08 +00:00
tonihei
29208ec1ed Define CueGroup.EMPTY_TIME_ZERO for convenience
We create an empty CueGroup in many places as default or
where none is needed. Instead, we can define a constant
for this purpose and reuse it.

PiperOrigin-RevId: 467944841
2022-10-19 18:30:47 +00:00
christosts
d3d5ffce91 Increase max sample size for HEVC.
Increase the estimated max sample size for HEVC by 2x, and set a minimum
size of 2MB. The 2MB will be applied for resolutions up to 1080p, after
which the new calculation takes effect. This is in par with the
platform's HEVC software decoder.

PiperOrigin-RevId: 467641494
2022-10-19 18:19:27 +00:00
tonihei
9a7fde8fde Add initial version of SimpleBasePlayer
This base class will simplify the implementation of custom
Player classes. The current version only supports
available commands and playWhenReady handling.

PiperOrigin-RevId: 467618021
2022-10-19 18:12:00 +00:00
huangdarwin
70972bbab0 HDR: Clarify tone mapping fallback.
createSupportedTransformationRequest is more accurate than
createFallbackTransformationRequest, as a TransformationRequest will be returned
regardless of whether any fallback is applied.

PiperOrigin-RevId: 466641277
2022-10-19 17:46:15 +00:00
huangdarwin
2d2926b8c0 HDR: Update experimental_setEnableHdrEditing javadoc.
Previously, this feature interpreted SDR signals as HDR when called. Now, only HDR
streams are interpreted as HDR, so the javadoc should be updated. Not yet removing
this method, as there are still some loose ends to finish up (ex. PQ support, e2e
tests).

PiperOrigin-RevId: 466425738
2022-10-19 17:38:55 +00:00
Googler
95a7dcaeda Remove times(1) from effect and transformer tests as it is the default.
PiperOrigin-RevId: 466324322
2022-10-19 17:27:57 +00:00
huangdarwin
7075f78eb2 HDR: Rename ColorInfo#isHdr to isTransferHdr.
While HDR is most closely tied to the color transfer (ex.
COLOR_TRANSFER_SDR is the only one explicitly mentioning dynamic
range), technically color spaces may be associated with HDR as well,
like BT.2020 commonly being used for HDR rather than BT.709 for SDR.

Therefore, it's more specific to mention just that the transfer is HDR.

PiperOrigin-RevId: 466316960
2022-10-19 17:24:19 +00:00
huangdarwin
e444eaa5b7 HDR: Input ColorInfo to the FrameProcessor.
This allows the GlEffectsFrameProcessor to later handle HLG and PQ
differently, or limited and full color range differently.

No functional change intended in this CL.

PiperOrigin-RevId: 466070764
2022-10-19 17:16:56 +00:00
hmzh
3aa99d5efb Add a custom MIME type for MIDI
PiperOrigin-RevId: 466032455
2022-10-19 17:13:18 +00:00
andrewlewis
273cf0866a Remove unused log tag constant
PiperOrigin-RevId: 466025324
2022-10-19 17:09:41 +00:00
andrewlewis
0e99d28a79 Throw if tone-mapping is requested but not enabled
In the case where this check fails, the downstream frame processor chain won't be able to handle the incoming (SDR) data anyway as we've already set it up for HDR.

PiperOrigin-RevId: 465584814
2022-10-19 16:58:47 +00:00
andrewlewis
d90d7d3078 Clarify SSIM request method name
`requestCalculateSsim` more clearly represents the intention of the caller.

Also rephrase the javadoc to simplify it and make it more precise.

PiperOrigin-RevId: 465575578
2022-10-19 16:55:11 +00:00
yschimke
c09b34a61b Make AudioTrackBufferSizeProvider public.
Was getting "java.lang.IllegalAccessError: Illegal class access" in debug app.

PiperOrigin-RevId: 465562541
2022-10-19 16:47:51 +00:00
andrewlewis
aafa31e996 Fix frame processor reference
PiperOrigin-RevId: 465562260
2022-10-19 16:44:08 +00:00
huangdarwin
e63d594f49 HDR: Add comment to clarify lack of support HDR10 for AVC
PiperOrigin-RevId: 465360186
2022-10-19 16:33:21 +00:00
claincly
a1be6d6eb3 Factor out RTP timestamp to sample time method
PiperOrigin-RevId: 465337074
2022-10-19 16:29:43 +00:00
yschimke
fe2b846552 Expose AudioOffload track state.
Adds a new event to AudioOffloadListener to get the offload state of the track, which indicates when software decoding is taking place.

PiperOrigin-RevId: 465264362
2022-10-19 16:26:07 +00:00
yschimke
2fbe1bbfb3 AudioOffload recovery.
Avoids disabling Offload on a write error, and instead relies on this being disabled on the AudioTrack init. It will no longer recover by disabling offload.

PiperOrigin-RevId: 465248917
2022-10-19 16:22:33 +00:00
andrewlewis
40a28d5aee Avoid encoder default color info when tone-mapping
The encoder color info [defaults](https://cs.android.com/android/platform/superproject/+/master:frameworks/av/media/libstagefright/foundation/ColorUtils.cpp;l=377;drc=891c19ccfa4953b5e5f7b87118e007b994d8074c) to a value that depends on the input resolution, but when tone-mapping is enabled we should always get BT.709 color space. Hard-code this constant for now to avoid behavior depending on the resolution. A future change should use the decoder output media format to populate the color info.

PiperOrigin-RevId: 465070378
2022-10-19 16:14:44 +00:00
claincly
13e9ec9959 Clean up NOP constructor lines in RTP readers
PiperOrigin-RevId: 465067191
2022-10-19 16:07:30 +00:00
Googler
245326a043 Clean up FrameProcessor TODOs.
PiperOrigin-RevId: 465044342
2022-10-19 16:03:47 +00:00
Googler
707b061838 Move effects functionality out of transformer to effects module.
PiperOrigin-RevId: 465038852
2022-10-19 16:00:13 +00:00
leonwind
23f34f8d08 Implement RGBA color matrices processor
* Transform frame colors using a defined RGBA Matrix to apply filters.

PiperOrigin-RevId: 464523581
2022-10-19 15:46:01 +00:00
leonwind
ea213545e0 Migrate Contrast asset to emulator generated one.
PiperOrigin-RevId: 464519030
2022-10-19 15:42:30 +00:00
huangdarwin
1a22bcf1e5 HDR: Have @C.ColorTransfer check consider NO_VALUE as SDR.
Format.NO_VALUE is a placeholder value for an invalid @C.ColorTransfer, used
for example when the decoder doesn't support this transfer function.

When encountering this invalid value, interpret this as COLOR_TRANSFER_SDR.

Confirmed locally that an exception is thrown when transcoding on p4head, and no exception is thrown when transcoding with this CL.

PiperOrigin-RevId: 464135080
2022-10-19 15:38:43 +00:00
Googler
06d41c2775 Move FrameProcessor and related interfaces to common.
This will allow effects preview in ExoPlayer to use the
Effect and FrameProcessor interface (and the interfaces
they depend on) without depending on transformer or the
future effects module.

PiperOrigin-RevId: 464060047
2022-10-19 15:34:54 +00:00
huangdarwin
810f451ec6 Transformer: Improve misc javadocs and exception logs.
PiperOrigin-RevId: 463971447
2022-10-19 15:27:24 +00:00
huangdarwin
44c42fef2a HDR: Centralize getting Colorinfo from MediaFormat
* Sets KEY_HDR_STATIC_INFO from MediaFormat in the DefaultCodec.
* Adds checks in mediaparser to ensure color space, range, and transfer are valid
  values.

PiperOrigin-RevId: 463921325
2022-10-19 15:23:34 +00:00
Marc Baechinger
9ed04ef842 Merge pull request #115 from ittiam-systems:rtp_vp9fix
PiperOrigin-RevId: 463852948
2022-10-19 15:19:39 +00:00
tonihei
3650c2970a Ignore stale events in StreamEventCallbackV29.
Despite unregistering the callback and clearing pending Handler
messages, the callback may still receive pending calls if they
are already triggered by the AudioTrack. Instead of asserting
that the track is correct, we should gracefully ignore stale
events.

PiperOrigin-RevId: 463851393
2022-10-19 15:15:35 +00:00
claincly
4d5d592c35 Fix VP8 reader fragment packet size
Merging c37a767fdf

PiperOrigin-RevId: 463826373
2022-10-19 15:11:52 +00:00
christosts
81d1dafec6 Support ApplicationVersion = 1 for HDR10+
MediaCodecRenderer allows passing HDR10+ out-of-bound metadata
with ApplicationVersion = 1.

PiperOrigin-RevId: 463822315
2022-10-19 15:08:06 +00:00
Marc Baechinger
3b5a53f1a6 Merge pull request #126 from stoyicker:cap_concurrent_remove_tasks
PiperOrigin-RevId: 463792127
2022-10-19 15:04:14 +00:00
Googler
bac323d346 Generalize frame processing error codes.
TransformationException error codes were previously tied to OpenGL
but other FrameProcessor implementations are possible. So this CL
renames the error codes.
Also, remove GL_INIT_FAILED error code, as FrameProcessor
implemenations may initialize resources on a background thread
after the factory method returns, so it's not obvious how to
distinguish between initialization failures and processing failures.

PiperOrigin-RevId: 463704902
2022-10-19 15:00:33 +00:00
Googler
66dde42989 Add Effect marker interface.
This allows non-GL effects to be passed to custom FrameProcessor
implementations.

PiperOrigin-RevId: 463696384
2022-10-19 14:56:44 +00:00
Googler
77f41d6fd4 Remove redundant interface from FinalMatrixTPWrapper.
PiperOrigin-RevId: 463646211
2022-10-19 14:53:00 +00:00
Googler
40e96d1534 Make FrameInfo and SurfaceInfo public.
Both are used in the public FrameProcessor interface, so they
should be public too.

PiperOrigin-RevId: 463454859
2022-10-19 14:49:07 +00:00
samrobinson
5f1ee86eb4 Add 8 new videos, representing lower resolutions.
Downsampled from 3840x2160 resolution videos using ffmpeg:

`ffmpeg -i <inputFilename> -filter:v scale=<width>:<height> -c:a copy <outputFilename>`

PiperOrigin-RevId: 463454628
2022-10-19 14:45:21 +00:00