16370 Commits

Author SHA1 Message Date
Googler
010ecec89d Reassign TODO to new bug.
PiperOrigin-RevId: 468672505
2022-10-19 19:06:04 +00:00
bachinger
d6442f63fa Add device info to CastPlayer
Issue: androidx/media#142
PiperOrigin-RevId: 468666737
2022-10-19 19:02:04 +00:00
Googler
4e7f9c575d Change onOutputFrameAvailable timestamp from nanos to micros.
Upstream timestamps from the decoder are also in microseconds,
so using microseconds here is consistent with that.

PiperOrigin-RevId: 468659099
2022-10-19 18:57:51 +00:00
hmzh
53218b50d4 Fix the command sample size given to trackOutput.
Remove the manual overwriting of Note ON events that have 0 velocity with Note OFF. JSyn handles this already.

- The implementation of "running status" means that the amount of bytes read from the file differ from the size of the sample that ends up in the decoder. The decoder sample contains the applied running status (status of previous event), which the file bytes don't contain.

PiperOrigin-RevId: 468537659
2022-10-19 18:53:55 +00:00
Googler
2c063546a1 Allow frame release to be controlled outside FrameProcessor.
Adds a method to FrameProcessor.Listener to be called when an
output frame is available and a method releaseOutputFrame in
FrameProcessor allowing the caller to trigger release of the
oldest available output frame at a given timestamp. Late frames
or frames with unset release times are dropped in the
FinalMatrixTransformationProcessorWrapper.

More than one output frame can become available before they are
released if the penultimate GlTextureProcessor is capable of producing
multiple output frames. Processing continues while waiting for
releaseOutputFrame to be called. Frame release tasks are prioritized
over other tasks.

PiperOrigin-RevId: 468473072
2022-10-19 18:50:08 +00:00
huangdarwin
32ee44805b HDR: Clamp YUV to RGB conversion.
Manually tested using transformer demo HLG videos. Before this CL, RGB values after the YUV to RGB conversion reached up to 1.025. After this CL, RGB values correctly clamp at 1.0.

PiperOrigin-RevId: 468426092
2022-10-19 18:45:48 +00:00
Googler
d11c3be952 Defensively cancel frame processing tasks on error.
FrameProcessingTaskExecutor should be released on error.
There can be a delay until this happens, so
FrameProcessingTaskExecutor will cancel any pending tasks
and drop new tasks until it is released.

PiperOrigin-RevId: 468171820
2022-10-19 18:41:54 +00:00
leonwind
c6a10ad227 Fix typo
PiperOrigin-RevId: 468152718
2022-10-19 18:38:09 +00:00
leonwind
977370e9db Compact a RGB Matrix chain into a singular RGB Matrix.
PiperOrigin-RevId: 468013019
2022-10-19 18:34:28 +00:00
tonihei
29208ec1ed Define CueGroup.EMPTY_TIME_ZERO for convenience
We create an empty CueGroup in many places as default or
where none is needed. Instead, we can define a constant
for this purpose and reuse it.

PiperOrigin-RevId: 467944841
2022-10-19 18:30:47 +00:00
huangdarwin
130a536287 HDR: Update limited range and add full range YUV to RGB color transforms.
PiperOrigin-RevId: 467910378
2022-10-19 18:26:58 +00:00
Googler
4d81a6db24 Allow high-priority tasks to be executed before other tasks.
This is needed as a pre-requisite for allowing MCVR to control
FrameProcessor frame release for previewing.

Submitting a high-priority task is conceptually different from
posting at the front of a single queue of tasks, as the high-priority
tasks are executed in FIFO order among themselves. This will ensure
that frame release tasks submitted in close succession are executed
in the order they are submitted but before any lower priority tasks.

PiperOrigin-RevId: 467675137
2022-10-19 18:23:14 +00:00
christosts
d3d5ffce91 Increase max sample size for HEVC.
Increase the estimated max sample size for HEVC by 2x, and set a minimum
size of 2MB. The 2MB will be applied for resolutions up to 1080p, after
which the new calculation takes effect. This is in par with the
platform's HEVC software decoder.

PiperOrigin-RevId: 467641494
2022-10-19 18:19:27 +00:00
tonihei
80b635e7ab Update CMake version for AV1 extension
The specified CMake version doesn't work with the latest
Android Studio releases. Updating to a more recent version
fixes the problem.

Issue: google/ExoPlayer#9933
PiperOrigin-RevId: 467634063
2022-10-19 18:15:44 +00:00
tonihei
9a7fde8fde Add initial version of SimpleBasePlayer
This base class will simplify the implementation of custom
Player classes. The current version only supports
available commands and playWhenReady handling.

PiperOrigin-RevId: 467618021
2022-10-19 18:12:00 +00:00
andrewlewis
ec7936f64d Increase pixel difference threshold
PiperOrigin-RevId: 467610621
2022-10-19 18:08:20 +00:00
leonwind
80f9ad9bf9 Refactor RgbaMatrix to RgbMatrix.
* Rename all Rgba instances to Rgb.
* Remove alpha value from the RGBA Matrices and apply the 4x4 matrix
only to the R, G, B channels.
* Restore the alpha from the input unchanged.

PiperOrigin-RevId: 467208888
2022-10-19 18:04:43 +00:00
Googler
06ce5ec738 Avoid spinning while queueing input to ExternalTextureProcessor.
This change adds ExternalTextureManager which implements
InputListener to only queue input frames to the
ExternalTextureProcessor when it is ready to accept an input
frame. This replaces the old retry-logic in GlEffectsFrameProcessor.

Before this change, the retrying in GlEffectFrameProcessor wasted
CPU time if input becomes available faster than the
ExternalTextureProcessor can process it.

PiperOrigin-RevId: 467177659
2022-10-19 18:00:59 +00:00
tonihei
6ea1d0ecf0 Add additional build flags for ffmpeg on "arm64-v8a"
From NDK 23.1.7779620 and above, the arm64-v8a ABI needs additional
build flags to correctly link the ffmpeg libraries.

Issue: google/ExoPlayer#9933
PiperOrigin-RevId: 467161973
2022-10-19 17:57:17 +00:00
tonihei
91fc5c4f3c Add timeout for ad to load.
In some cases, the IMA SDK fails to call the expected loadAd
event to load the next ad to play. This is (potentially) the
only remaining case where playback can get stuck due to missing
calls from IMA as the player doesn't even have a MediaSource at
this stage and is only waiting for IMA to provide the ad URL.

We can reuse the existing adPreloadTimeoutMs that was added for
a similar purpose (when preloading the first ad in the group).
The JavaDoc matches this purpose as well and the default timeout
is appropriate since we expect to get the loadAd call immediately.

Issue: google/ExoPlayer#10510
PiperOrigin-RevId: 466953617
2022-10-19 17:53:39 +00:00
Googler
43aa89aa2e Fix ChainingGlTextureProcessorListener field name.
PiperOrigin-RevId: 466735554
2022-10-19 17:49:53 +00:00
huangdarwin
70972bbab0 HDR: Clarify tone mapping fallback.
createSupportedTransformationRequest is more accurate than
createFallbackTransformationRequest, as a TransformationRequest will be returned
regardless of whether any fallback is applied.

PiperOrigin-RevId: 466641277
2022-10-19 17:46:15 +00:00
Googler
9c366b3cfd Avoid spinning in between intermediate texture processors.
This change adds a new method onReadyToAcceptInputFrame to
GlTextureProcesssor.InputListener and changes maybeQueueInputFrame
to queueInputFrame, removing the boolean return value.
This avoids the re-trying in ChainingGlTextureProcessorListener
by allowing it to only feed frames from the producing to the consuming
GlTextureProcessor when there is capacity.

MediaPipeProcessor still needs re-trying when processing isn't 1:1.

PiperOrigin-RevId: 466626369
2022-10-19 17:42:37 +00:00
huangdarwin
2d2926b8c0 HDR: Update experimental_setEnableHdrEditing javadoc.
Previously, this feature interpreted SDR signals as HDR when called. Now, only HDR
streams are interpreted as HDR, so the javadoc should be updated. Not yet removing
this method, as there are still some loose ends to finish up (ex. PQ support, e2e
tests).

PiperOrigin-RevId: 466425738
2022-10-19 17:38:55 +00:00
huangdarwin
051dee68e3 Demo: Add legacy external storage permission for intents.
Without this permission, files in `/sdcard` or other directories cannot be read by the demo, as they're not in the transformer demo's scoped storage container.

For more information, see https://developer.android.com/training/data-storage/use-cases

Tested by uninstalling and re-installing the demo app, granting permission by
starting a transformation, and launching an intent using a local file in `/sdcard`
to start another transformation. Without this CL, this threw an error, and with
this CL it succeeded.

PiperOrigin-RevId: 466399023
2022-10-19 17:35:20 +00:00
hmzh
96a2b149b1 Add JSyn library build instructions for the MIDI module
PiperOrigin-RevId: 466370644
2022-10-19 17:31:44 +00:00
Googler
95a7dcaeda Remove times(1) from effect and transformer tests as it is the default.
PiperOrigin-RevId: 466324322
2022-10-19 17:27:57 +00:00
huangdarwin
7075f78eb2 HDR: Rename ColorInfo#isHdr to isTransferHdr.
While HDR is most closely tied to the color transfer (ex.
COLOR_TRANSFER_SDR is the only one explicitly mentioning dynamic
range), technically color spaces may be associated with HDR as well,
like BT.2020 commonly being used for HDR rather than BT.709 for SDR.

Therefore, it's more specific to mention just that the transfer is HDR.

PiperOrigin-RevId: 466316960
2022-10-19 17:24:19 +00:00
Googler
a2166a4142 Split GlTextureProcessor.Listener into input/output/error listener.
This simplifies ChainingGlTextureProcessor as it now only connects a
consuming and a producing GlTextureProcessor rather than a previous,
current, and next GlTextureProcessor.

Also use default no-op implementations of the listeners in
SingleFrameGlTextureProcessor and MediaPipeProcessor to avoid
null-checks.

PiperOrigin-RevId: 466301642
2022-10-19 17:20:38 +00:00
huangdarwin
e444eaa5b7 HDR: Input ColorInfo to the FrameProcessor.
This allows the GlEffectsFrameProcessor to later handle HLG and PQ
differently, or limited and full color range differently.

No functional change intended in this CL.

PiperOrigin-RevId: 466070764
2022-10-19 17:16:56 +00:00
hmzh
3aa99d5efb Add a custom MIME type for MIDI
PiperOrigin-RevId: 466032455
2022-10-19 17:13:18 +00:00
andrewlewis
273cf0866a Remove unused log tag constant
PiperOrigin-RevId: 466025324
2022-10-19 17:09:41 +00:00
hmzh
4725cff70e Add support for multiple mid-track tempo changes
PiperOrigin-RevId: 466018462
2022-10-19 17:06:02 +00:00
leonwind
650a9d21ec Switch @param to @code tags in RgbAdjustment.
PiperOrigin-RevId: 465591877
2022-10-19 17:02:18 +00:00
andrewlewis
0e99d28a79 Throw if tone-mapping is requested but not enabled
In the case where this check fails, the downstream frame processor chain won't be able to handle the incoming (SDR) data anyway as we've already set it up for HDR.

PiperOrigin-RevId: 465584814
2022-10-19 16:58:47 +00:00
andrewlewis
d90d7d3078 Clarify SSIM request method name
`requestCalculateSsim` more clearly represents the intention of the caller.

Also rephrase the javadoc to simplify it and make it more precise.

PiperOrigin-RevId: 465575578
2022-10-19 16:55:11 +00:00
Googler
656753b809 Corrected ordering of javadoc params
PiperOrigin-RevId: 482231370
(cherry picked from commit 2c61dd9c95e64d0e137d1a4775cd23bc9c0ac42f)
2022-10-19 16:53:37 +00:00
Googler
6e4fc47522 Fix ConcurrentHashMap usage in demo MediaPipeProcessor.
PiperOrigin-RevId: 465563540
2022-10-19 16:51:30 +00:00
yschimke
c09b34a61b Make AudioTrackBufferSizeProvider public.
Was getting "java.lang.IllegalAccessError: Illegal class access" in debug app.

PiperOrigin-RevId: 465562541
2022-10-19 16:47:51 +00:00
ibaker
b3cbebb036 Remove leading space from RELEASENOTES.md
PiperOrigin-RevId: 482229341
(cherry picked from commit f9a59ab3b51f56f2b29355180518c5e990b01b2e)
2022-10-19 16:45:30 +00:00
andrewlewis
aafa31e996 Fix frame processor reference
PiperOrigin-RevId: 465562260
2022-10-19 16:44:08 +00:00
Googler
6da0a35434 Use multiple output frames in transformer MediaPipe demo pre API 23.
PiperOrigin-RevId: 465545764
2022-10-19 16:40:29 +00:00
leonwind
6502fce634 Add RgbAdjustment class to build RgbaMatrices
* Add RgbaMatrix interface implementation.
* Add Builder class for easy adjustments.
* Adjust existing RgbaMatrixPixelTests to use new RgbAdjustment class.

PiperOrigin-RevId: 465545429
2022-10-19 16:36:55 +00:00
huangdarwin
e63d594f49 HDR: Add comment to clarify lack of support HDR10 for AVC
PiperOrigin-RevId: 465360186
2022-10-19 16:33:21 +00:00
claincly
a1be6d6eb3 Factor out RTP timestamp to sample time method
PiperOrigin-RevId: 465337074
2022-10-19 16:29:43 +00:00
yschimke
fe2b846552 Expose AudioOffload track state.
Adds a new event to AudioOffloadListener to get the offload state of the track, which indicates when software decoding is taking place.

PiperOrigin-RevId: 465264362
2022-10-19 16:26:07 +00:00
yschimke
2fbe1bbfb3 AudioOffload recovery.
Avoids disabling Offload on a write error, and instead relies on this being disabled on the AudioTrack init. It will no longer recover by disabling offload.

PiperOrigin-RevId: 465248917
2022-10-19 16:22:33 +00:00
leonwind
30fab8c008 Remove clamp function in contrast fragment shader.
* OpenGL automatically clamps the output colors to the [0, 1] interval.

PiperOrigin-RevId: 465071999
2022-10-19 16:18:51 +00:00
andrewlewis
40a28d5aee Avoid encoder default color info when tone-mapping
The encoder color info [defaults](https://cs.android.com/android/platform/superproject/+/master:frameworks/av/media/libstagefright/foundation/ColorUtils.cpp;l=377;drc=891c19ccfa4953b5e5f7b87118e007b994d8074c) to a value that depends on the input resolution, but when tone-mapping is enabled we should always get BT.709 color space. Hard-code this constant for now to avoid behavior depending on the resolution. A future change should use the decoder output media format to populate the color info.

PiperOrigin-RevId: 465070378
2022-10-19 16:14:44 +00:00
Googler
bbb7b1c0bc Fix saving previous debugSurfaceView in FinalMatrixTPWrapper.
PiperOrigin-RevId: 465067306
2022-10-19 16:11:06 +00:00