14674 Commits

Author SHA1 Message Date
huangdarwin
da10c96229 HDR: Clarify tone mapping fallback.
createSupportedTransformationRequest is more accurate than
createFallbackTransformationRequest, as a TransformationRequest will be returned
regardless of whether any fallback is applied.

PiperOrigin-RevId: 466641277
(cherry picked from commit c5a5364673749cb737be3df92c9521fc15207658)
2022-08-10 11:25:56 +00:00
Googler
6b4d174cde Avoid spinning in between intermediate texture processors.
This change adds a new method onReadyToAcceptInputFrame to
GlTextureProcesssor.InputListener and changes maybeQueueInputFrame
to queueInputFrame, removing the boolean return value.
This avoids the re-trying in ChainingGlTextureProcessorListener
by allowing it to only feed frames from the producing to the consuming
GlTextureProcessor when there is capacity.

MediaPipeProcessor still needs re-trying when processing isn't 1:1.

PiperOrigin-RevId: 466626369
(cherry picked from commit 97e6a86d2b1f72de9f95658f90ea40d1e1e4fac2)
2022-08-10 09:56:24 +00:00
huangdarwin
f83753d4e0 HDR: Update experimental_setEnableHdrEditing javadoc.
Previously, this feature interpreted SDR signals as HDR when called. Now, only HDR
streams are interpreted as HDR, so the javadoc should be updated. Not yet removing
this method, as there are still some loose ends to finish up (ex. PQ support, e2e
tests).

PiperOrigin-RevId: 466425738
(cherry picked from commit 05fcda510a1dc8ab0a44487f3ddfff7fc2b617e7)
2022-08-09 18:56:08 +00:00
huangdarwin
b89320521a Demo: Add legacy external storage permission for intents.
Without this permission, files in `/sdcard` or other directories cannot be read by the demo, as they're not in the transformer demo's scoped storage container.

For more information, see https://developer.android.com/training/data-storage/use-cases

Tested by uninstalling and re-installing the demo app, granting permission by
starting a transformation, and launching an intent using a local file in `/sdcard`
to start another transformation. Without this CL, this threw an error, and with
this CL it succeeded.

PiperOrigin-RevId: 466399023
(cherry picked from commit ec6ed0d55f59c9387aa09b1514bea9951a6206f8)
2022-08-09 17:28:31 +00:00
hmzh
2ea29916c3 Add JSyn library build instructions for the MIDI module
PiperOrigin-RevId: 466370644
(cherry picked from commit 3bd91707dc25e668a5426a5ec4869d4d0b8749b8)
2022-08-09 15:44:44 +00:00
Googler
ccef3950ca Remove times(1) from effect and transformer tests as it is the default.
PiperOrigin-RevId: 466324322
(cherry picked from commit 1bdc252d2cfac5499306584fd68a2b1f33f05cc4)
2022-08-09 12:08:39 +00:00
huangdarwin
00749d9d26 HDR: Rename ColorInfo#isHdr to isTransferHdr.
While HDR is most closely tied to the color transfer (ex.
COLOR_TRANSFER_SDR is the only one explicitly mentioning dynamic
range), technically color spaces may be associated with HDR as well,
like BT.2020 commonly being used for HDR rather than BT.709 for SDR.

Therefore, it's more specific to mention just that the transfer is HDR.

PiperOrigin-RevId: 466316960
(cherry picked from commit 35161c748956d96e8e83cee007143615ef6f9d07)
2022-08-09 11:28:31 +00:00
Googler
cdd47ff703 Split GlTextureProcessor.Listener into input/output/error listener.
This simplifies ChainingGlTextureProcessor as it now only connects a
consuming and a producing GlTextureProcessor rather than a previous,
current, and next GlTextureProcessor.

Also use default no-op implementations of the listeners in
SingleFrameGlTextureProcessor and MediaPipeProcessor to avoid
null-checks.

PiperOrigin-RevId: 466301642
(cherry picked from commit 7dc05edbab233dcd631a4a78ad65f1c30a1e9734)
2022-08-09 10:05:14 +00:00
huangdarwin
37274c91de HDR: Input ColorInfo to the FrameProcessor.
This allows the GlEffectsFrameProcessor to later handle HLG and PQ
differently, or limited and full color range differently.

No functional change intended in this CL.

PiperOrigin-RevId: 466070764
(cherry picked from commit 536d42c865fd692c1d1799082077906287a2c3f0)
2022-08-08 16:55:22 +00:00
hmzh
dc29b44f05 Add a custom MIME type for MIDI
PiperOrigin-RevId: 466032455
(cherry picked from commit 579f25802a7d51b712c80e2045a76697be0a72e8)
2022-08-08 14:02:08 +00:00
andrewlewis
5385cc0422 Remove unused log tag constant
PiperOrigin-RevId: 466025324
(cherry picked from commit 2315a39f5240a422e915c4953d159db9eb439b27)
2022-08-08 13:20:34 +00:00
hmzh
dbf32af099 Add support for multiple mid-track tempo changes
PiperOrigin-RevId: 466018462
(cherry picked from commit 22b00d0e00d695960fa609fee8a6cb3986bb3e61)
2022-08-08 12:35:28 +00:00
leonwind
1bbb5df410 Switch @param to @code tags in RgbAdjustment.
PiperOrigin-RevId: 465591877
(cherry picked from commit fe0cf0528318abf1a9a7a8f75bcce402235f961a)
2022-08-05 17:20:54 +00:00
andrewlewis
08bbb79c21 Throw if tone-mapping is requested but not enabled
In the case where this check fails, the downstream frame processor chain won't be able to handle the incoming (SDR) data anyway as we've already set it up for HDR.

PiperOrigin-RevId: 465584814
(cherry picked from commit 2bd1a93d27fbc1615c55edb200917f5d5a1891b4)
2022-08-05 16:49:26 +00:00
andrewlewis
99202c5568 Clarify SSIM request method name
`requestCalculateSsim` more clearly represents the intention of the caller.

Also rephrase the javadoc to simplify it and make it more precise.

PiperOrigin-RevId: 465575578
(cherry picked from commit 2eb59fa1af823ff66fcf384f567a92e55c35b683)
2022-08-05 16:02:07 +00:00
hschlueter
bb57996c53 Fix ConcurrentHashMap usage in demo MediaPipeProcessor.
PiperOrigin-RevId: 465563540
(cherry picked from commit c95bfe325d75c991937fb8e2ecda58e59b6ca8df)
2022-08-05 14:56:54 +00:00
yschimke
d523786aef Make AudioTrackBufferSizeProvider public.
Was getting "java.lang.IllegalAccessError: Illegal class access" in debug app.

PiperOrigin-RevId: 465562541
(cherry picked from commit 1cfde16efdca6aa6d217e90dedb67a7916c315a7)
2022-08-05 14:51:07 +00:00
andrewlewis
5dd796e14d Fix frame processor reference
PiperOrigin-RevId: 465562260
(cherry picked from commit 16a325217abf0b52eb103b86164000c1e93cb1b3)
2022-08-05 14:49:23 +00:00
hschlueter
fa1a7eb51e Use multiple output frames in transformer MediaPipe demo pre API 23.
PiperOrigin-RevId: 465545764
(cherry picked from commit 51a1a879faadf749348902c12b382b42d54cd202)
2022-08-05 12:58:48 +00:00
leonwind
d34ed7a5e1 Add RgbAdjustment class to build RgbaMatrices
* Add RgbaMatrix interface implementation.
* Add Builder class for easy adjustments.
* Adjust existing RgbaMatrixPixelTests to use new RgbAdjustment class.

PiperOrigin-RevId: 465545429
(cherry picked from commit a76dbfd363e2d167b13d2bcd978979bcfd69b071)
2022-08-05 12:56:26 +00:00
huangdarwin
7eab760b18 HDR: Add comment to clarify lack of support HDR10 for AVC
PiperOrigin-RevId: 465360186
(cherry picked from commit 348662283a60c88e6c5a0deef9307da75161b241)
2022-08-04 18:18:18 +00:00
claincly
e57a36bb20 Factor out RTP timestamp to sample time method
PiperOrigin-RevId: 465337074
(cherry picked from commit 80ec64696e22856ce324ef03535754f838a49778)
2022-08-04 16:49:03 +00:00
tonihei
35565cfea7 Update README for beta release.
PiperOrigin-RevId: 465318320
(cherry picked from commit afc928330b64424a6db1958910b688ee16dcebc5)
2022-08-04 15:17:56 +00:00
yschimke
7a8f348228 Expose AudioOffload track state.
Adds a new event to AudioOffloadListener to get the offload state of the track, which indicates when software decoding is taking place.

PiperOrigin-RevId: 465264362
(cherry picked from commit 7893531888608555fb09e77f12897752650131d5)
2022-08-04 09:47:41 +00:00
yschimke
e5f17f44d5 AudioOffload recovery.
Avoids disabling Offload on a write error, and instead relies on this being disabled on the AudioTrack init. It will no longer recover by disabling offload.

PiperOrigin-RevId: 465248917
(cherry picked from commit a10af8ecda584ea0d2bbb2616296b60aace8d638)
2022-08-04 08:00:57 +00:00
leonwind
46f267700f Remove clamp function in contrast fragment shader.
* OpenGL automatically clamps the output colors to the [0, 1] interval.

PiperOrigin-RevId: 465071999
(cherry picked from commit 2347db5d7484b161f6389846e6cde5b0ac1ed7a6)
2022-08-03 16:14:30 +00:00
andrewlewis
f8aaed1ac7 Avoid encoder default color info when tone-mapping
The encoder color info [defaults](https://cs.android.com/android/platform/superproject/+/master:frameworks/av/media/libstagefright/foundation/ColorUtils.cpp;l=377;drc=891c19ccfa4953b5e5f7b87118e007b994d8074c) to a value that depends on the input resolution, but when tone-mapping is enabled we should always get BT.709 color space. Hard-code this constant for now to avoid behavior depending on the resolution. A future change should use the decoder output media format to populate the color info.

PiperOrigin-RevId: 465070378
(cherry picked from commit 3c4ee51e10fe0a2ea495f00a3fcd98d6cb83b561)
2022-08-03 16:06:45 +00:00
hschlueter
110c442758 Fix saving previous debugSurfaceView in FinalMatrixTPWrapper.
PiperOrigin-RevId: 465067306
(cherry picked from commit a28a508d2dd10ac2858c334a1a0fe0ff1350e98c)
2022-08-03 15:52:24 +00:00
claincly
076218be9d Clean up NOP constructor lines in RTP readers
PiperOrigin-RevId: 465067191
(cherry picked from commit ed60c885f2c09bcc8484eeeb340a96559c7ad918)
2022-08-03 15:51:48 +00:00
hschlueter
89edfbc12b Clean up FrameProcessor TODOs.
PiperOrigin-RevId: 465044342
(cherry picked from commit c5d1940fbab430ede4b158300bbb381c032498bf)
2022-08-03 13:53:53 +00:00
hschlueter
8b9203a3ea Move effects functionality out of transformer to effects module.
PiperOrigin-RevId: 465038852
(cherry picked from commit 22725ddfa8f5c39c83b792a176339700364b724e)
2022-08-03 13:19:15 +00:00
hschlueter
93bb191cb1 Create effect module.
PiperOrigin-RevId: 464767396
(cherry picked from commit 4d09ca66986a3c862b09f42a06a5bb031dce46f8)
2022-08-02 12:36:47 +00:00
leonwind
2c4e946f71 Implement RGBA color matrices processor
* Transform frame colors using a defined RGBA Matrix to apply filters.

PiperOrigin-RevId: 464523581
(cherry picked from commit d15f2ed388e59e1b378440470347ee6f7f00ed94)
2022-08-01 13:47:21 +00:00
leonwind
1ef01adc74 Migrate Contrast asset to emulator generated one.
PiperOrigin-RevId: 464519030
(cherry picked from commit ae1915ea1acea1181198b71ae654830c7512337b)
2022-08-01 13:14:49 +00:00
huangdarwin
c34e41950b HDR: Have @C.ColorTransfer check consider NO_VALUE as SDR.
Format.NO_VALUE is a placeholder value for an invalid @C.ColorTransfer, used
for example when the decoder doesn't support this transfer function.

When encountering this invalid value, interpret this as COLOR_TRANSFER_SDR.

Confirmed locally that an exception is thrown when transcoding on p4head, and no exception is thrown when transcoding with this CL.

PiperOrigin-RevId: 464135080
(cherry picked from commit 687a50e9b313e5db37a0b32f5f40559252ab6031)
2022-07-29 19:33:44 +00:00
hschlueter
de4c2e08e1 Move FrameProcessor and related interfaces to common.
This will allow effects preview in ExoPlayer to use the
Effect and FrameProcessor interface (and the interfaces
they depend on) without depending on transformer or the
future effects module.

PiperOrigin-RevId: 464060047
(cherry picked from commit 480c129c54777780f298d7e5dbd9fa23a371f660)
2022-07-29 12:54:30 +00:00
bachinger
1bee4f86fb Add interface version of MediaSessionStub
PiperOrigin-RevId: 464052708
(cherry picked from commit a28b3ef77843bb75938b2ca738ebedf0c29c1129)
2022-07-29 11:52:06 +00:00
leonwind
1b5e3ceb32 Add slider option for contrast in demo app
PiperOrigin-RevId: 464050072
(cherry picked from commit 3fae9df8a9a1ab844d65a2336fc75a57e53d78d2)
2022-07-29 11:30:44 +00:00
tonihei
7411a31e7b Merge pull request #109 from tzugen:patch-1
PiperOrigin-RevId: 464045351
(cherry picked from commit 61e4f92310091737b7107662d52ef8e13319f6e5)
2022-08-08 08:28:35 +00:00
huangdarwin
071f0a5a0d Transformer: Improve misc javadocs and exception logs.
PiperOrigin-RevId: 463971447
(cherry picked from commit 534740fd46f5ec9465215eaaa03a764eebab4ac4)
2022-07-29 01:16:26 +00:00
bachinger
8c560d2115 Add interface version of MediaControllerStub
PiperOrigin-RevId: 463930162
(cherry picked from commit c8089ead42d940c550955f7786447ad87784b9df)
2022-07-28 21:45:49 +00:00
huangdarwin
db1cad911c HDR: Centralize getting Colorinfo from MediaFormat
* Sets KEY_HDR_STATIC_INFO from MediaFormat in the DefaultCodec.
* Adds checks in mediaparser to ensure color space, range, and transfer are valid
  values.

PiperOrigin-RevId: 463921325
(cherry picked from commit 65a2a3a0a073c6dd8b47a044609f4be1abdd19c9)
2022-07-28 21:07:43 +00:00
tonihei
0c634543ce Merge pull request #115 from ittiam-systems:rtp_vp9fix
PiperOrigin-RevId: 463852948
(cherry picked from commit 92816023f16ac580a038e970b41aba7ae18f6055)
2022-08-08 08:23:56 +00:00
christosts
60b334cd01 Minor: Change URL in test
Use a URL with a `.test` tld (RFC 2606) to make it clear
it's an testing URL.

PiperOrigin-RevId: 463852174
(cherry picked from commit 4713c7fea835a8b73b41f6e4723f141828814e44)
2022-07-28 16:12:28 +00:00
tonihei
05658504d6 Ignore stale events in StreamEventCallbackV29.
Despite unregistering the callback and clearing pending Handler
messages, the callback may still receive pending calls if they
are already triggered by the AudioTrack. Instead of asserting
that the track is correct, we should gracefully ignore stale
events.

PiperOrigin-RevId: 463851393
(cherry picked from commit e331bf2c53c2ff7a3c998ccd3a58eb4b7c948fcc)
2022-07-28 16:08:21 +00:00
claincly
d20a1c7f29 Fix VP8 reader fragment packet size
Merging c37a767fdf

PiperOrigin-RevId: 463826373
(cherry picked from commit e848352a3e590de740bd88d816c22e05cca9cc07)
2022-07-28 13:50:54 +00:00
christosts
8779078088 Support ApplicationVersion = 1 for HDR10+
MediaCodecRenderer allows passing HDR10+ out-of-bound metadata
with ApplicationVersion = 1.

PiperOrigin-RevId: 463822315
(cherry picked from commit 95becec1e2e664f360bad731f750dcd05f3d79b0)
2022-07-28 13:24:08 +00:00
tonihei
561b83e282 Delay unbinding the controller until pending commands are handled.
Once a controller bound to a MediaSessionService unbinds, this service
may be destroyed immediately if this was the last bound controller and
the service isn't started in the foreground.

At the time of releasing the controller, there may still be pending
commands that need to be handled by the session service. These commands
may cause the session service to post a foreground notification to
keep it alive. So to avoid the destruction of the service while these
commands are still handled, we need to keep the controller bound.

We also add a timeout in case the session tasks are never completed
by the session implementation. In case the controller is destroyed,
the unbinding happens automatically by the system.

PiperOrigin-RevId: 463802220
(cherry picked from commit ac42b593cc3280b866d42cdc11d57bb031b39757)
2022-07-28 11:10:36 +00:00
tonihei
4eff5e5172 Merge pull request #126 from stoyicker:cap_concurrent_remove_tasks
PiperOrigin-RevId: 463792127
(cherry picked from commit 7f08ab729ad571247d2e65870eee06166b3bfdef)
2022-08-08 08:18:45 +00:00
hschlueter
427a7a1ff9 Generalize frame processing error codes.
TransformationException error codes were previously tied to OpenGL
but other FrameProcessor implementations are possible. So this CL
renames the error codes.
Also, remove GL_INIT_FAILED error code, as FrameProcessor
implemenations may initialize resources on a background thread
after the factory method returns, so it's not obvious how to
distinguish between initialization failures and processing failures.

PiperOrigin-RevId: 463704902
(cherry picked from commit 0f96d8615d28cf50f3efa0c34fb31e01fa0d491c)
2022-07-27 23:21:31 +00:00