1208 Commits

Author SHA1 Message Date
sheenachhabra
e0d6f67dd9 Move NAL unit related methods from common to container module
PiperOrigin-RevId: 532750099
2023-05-17 16:32:46 +01:00
tofunmi
88642587ac Update mixed input test to include images of different aspect ratios
Makes this test a little more thorough.

PiperOrigin-RevId: 532386515
2023-05-17 08:35:36 +01:00
huangdarwin
62afbe87bb Test: Suppress SSIM timeouts on galaxy fold/flip.
This is a failure only in SSIM, so it seems unlikely we'll prioritize this over
other work or bugs soon. Suppress test failures to reduce triage burden.

PiperOrigin-RevId: 532200729
2023-05-16 09:59:36 +01:00
huangdarwin
07ec1eaa48 Effect: Multiple Texture output
Allow the VideoFrameProcessor to output multiple textures at a time, so that
lifetime of textures is up to the consumer calling VFP.releaseOutputFrame.

The FinalShaderProgramWrapper also has a new maxCapacity limit added, to ensure
the a reasonable amount of textures is used and avoid using up memory.

PiperOrigin-RevId: 532094256
2023-05-16 09:49:52 +01:00
kimvde
6dfb387117 Refactor SequenceAssetLoader release
In the past, the SequenceAssetLoader was released in TransformerInternal
when the export ended.

fc539da061 was made to release the SequenceAssetLoader earlier, when
loading ended. This was causing player release timeouts because the last
AssetLoader in the sequence was released before the SamplePipelines (see
0b40bc37ab for more information).

The code that was releasing the SequenceAssetLoader was first commented
out because we didn't have an understanding of what was happening.

This change removes the early SequenceAssetLoader release all together.
It doesn't have any effect as this code was already commented out.

PiperOrigin-RevId: 532065673
2023-05-16 09:43:36 +01:00
huangdarwin
2736b11845 Transformer: Add latest input format to DebugTraceUtil
This can provide more information to help debug muxer errors

PiperOrigin-RevId: 531521974
2023-05-15 10:39:57 +01:00
sheenachhabra
867355fdc5 Rollback of 925aa34e13
*** Original commit ***

Rollback of 65d5132f76

*** Original commit ***

Create InAppMuxer in transformer

To use the InAppMuxer, the client needs to pass InAppMuxer Factory.

***

***

PiperOrigin-RevId: 531470081
2023-05-12 13:20:31 +00:00
huangdarwin
5313763eb3 Test: Filter test more precisely, with model instead of manufacturer
PiperOrigin-RevId: 531233480
2023-05-12 10:10:15 +01:00
huangdarwin
674885cfa5 Test: Suppress failing 8k24 SSIM bug.
PiperOrigin-RevId: 531193242
2023-05-12 10:07:55 +01:00
kimvde
203450d244 Small fixes in TextureAssetLoader
The texture input tests in TransformerEndToEndTest were not passing on
Pixel 7. Implemented a fix and fixed other minor threading issues I
spotted while looking at the code.

PiperOrigin-RevId: 531141659
2023-05-11 10:07:43 +00:00
kimvde
e8072ca2c7 Move VideoFrameProcessorFactory setter to Transformer
PiperOrigin-RevId: 531123743
2023-05-11 10:06:21 +00:00
huangdarwin
82ede47398 media3: Capitalize "MIME" for MIME types.
Find and replace "mime" to "MIME", where appropriate, throughout media3.

PiperOrigin-RevId: 531122121
2023-05-11 10:05:01 +00:00
Googler
925aa34e13 Rollback of 65d5132f76
*** Original commit ***

Create InAppMuxer in transformer

To use the InAppMuxer, the client needs to pass InAppMuxer Factory.

***

PiperOrigin-RevId: 531056436
2023-05-11 10:03:35 +00:00
tofunmi
7fea435b7e Transformer: Support Texture asset loading
PiperOrigin-RevId: 530888319
2023-05-11 10:01:00 +00:00
Googler
ecaf219d74 Adding mime type checking to Transformer to identify images.
PiperOrigin-RevId: 530711523
2023-05-11 09:58:15 +00:00
sheenachhabra
65d5132f76 Create InAppMuxer in transformer
To use the InAppMuxer, the client needs to pass InAppMuxer Factory.

PiperOrigin-RevId: 530684007
2023-05-11 09:56:46 +00:00
kimvde
dc4c6daf0b Fix HDR tone mapping stuck on SM device
Tone-mapping an HDR video with MediaCodec on sm-s908u1 was timing out.
The reason for that is that the decoder was dropping frames, and the
ExternalTextureManager was therefore never propagating the end-of-stream
signal.

There was already a workaround for a similar issue but restricted to
sm-f936b. Removed the model check as the bug is probably present on more
devices.

PiperOrigin-RevId: 530639437
2023-05-11 09:53:25 +00:00
kimvde
1105f194ca Avoid swallowing exceptions when switching AssetLoader
Exceptions thrown when switching AssetLoader were not reported through
the error listener. Therefore, the resources were not released and the
export was not ending.

PiperOrigin-RevId: 530626300
2023-05-11 09:50:43 +00:00
tofunmi
7875131e2a Effect: Support input via texture ID
PiperOrigin-RevId: 530624195
2023-05-11 09:49:26 +00:00
huangdarwin
2db2de5993 Transformer: Add codec support for Dolby Vision HDR video
Allow use of H265/H264 codecs for Dolby Vision video.

Also, reflow ExoPlayer code to use this new utility class

PiperOrigin-RevId: 530619388
2023-05-11 09:47:52 +00:00
kimvde
129a6e0cc6 Fix encoder expecting HDR while tone-mapping requested
If tone-mapping was requested for a device supporting HDR encoding,
isToneMapped was false in VideoSamplePipeline.EncoderWrapper. This was
causing the encoder to expect HDR.

Also did some renamings to improve readability

PiperOrigin-RevId: 530584010
2023-05-11 09:45:09 +00:00
tofunmi
b1ebd0e2a6 Change order of operations in videoSamplePipeline
similar error to 9baa6f6be5L `registerInputStream` sets the textureManager, so reordering makes sure the textureManager is set before you set the frame info. This is important for texture input, where the frame info provides the width and height.

PiperOrigin-RevId: 530579584
2023-05-11 09:43:40 +00:00
tofunmi
477994461d Image transcoding: Add support for heic/heif image formats
PiperOrigin-RevId: 530578549
2023-05-11 09:42:12 +00:00
andrewlewis
f0ff57882a Remove stray stuck debug logging line
PiperOrigin-RevId: 530314222
2023-05-11 09:39:17 +00:00
kimvde
30957df635 Remove B-frame setting from DefaultEncoderFactory
This setting is buggy on some devices (for example sm-n960f)

PiperOrigin-RevId: 529712770
2023-05-05 16:54:27 +00:00
claincly
83b5f824f8 Add a tracing utility for debugging video related muxer timeout
PiperOrigin-RevId: 529632886
2023-05-05 16:45:38 +00:00
claincly
213740c456 Enable switching input types via InputMultiplexer.
PiperOrigin-RevId: 529624205
2023-05-05 16:43:58 +00:00
tofunmi
3011a29940 Update InternetConnection check to skip check when uri scheme is null
When we created androidTests, in the past, they always had a URI pointing to a resource, therefore we always had a URI scheme. With texture input, this will not longer be the case (EditedMediaItems's may have URI.EMPTY, which have a null scheme) so we need to check for this so tests don't falsely fail.

PiperOrigin-RevId: 528848411
2023-05-03 17:12:29 +01:00
huangdarwin
24343f55af effect: Rename VFP frame release to render.
renderOutputFrame actually renders frames to an output surface. We'll soon have
a releaseOutputFrame method, that would release resources associated with an
output time, so rename this to disambiguate the two methods.

Also rename onOutputFrameAvailable to onOutputFrameAvailableForRendering, to
make it clear this is not available for "release"

This change should be a renaming-only change and have no functional differences.

PiperOrigin-RevId: 527844947
2023-05-03 17:00:48 +01:00
tofunmi
0902fad55d Effect: split ending frame Processing and getting the output bitmap in tests
PiperOrigin-RevId: 527554408
2023-05-03 16:54:53 +01:00
claincly
12cac0d69f Add a listener once one MediaItem is fully processed
Add `VideoFrameProcessor.registerInputStream()` to signal a new type of input.

And `InputHandler.signalEndOfCurrentInputStream()` to signal to `InputHandler`
partial input stream completion.

Fully processed means after FinalShaderProgramWrapper releases the last frame.

PiperOrigin-RevId: 527356646
2023-04-27 12:28:40 +01:00
huangdarwin
3db4205ac6 Test: Add @Ignore to Analysis tests
In ASwB, all Transformer tests can be run by right-clicking on the project and
clicking "Run all tests". We cannot, however, select only some specific test
files within a project (ex. all non-analysis tests) to run tests on.

Add @Ignore to analysis tests, which are not intended to be run anyways
when determining whether Transformer is working on some device. These
tests also don't have proper skipping logic when a device doesn't support a
format, so they can't be run effectively on low-end devices anyways.

This eases manual testing, for example when debugging whether tests all pass
on a device.

When analysis tests are desired to be run, it should be easy to comment out
the @Ignore.

PiperOrigin-RevId: 527289600
2023-04-27 12:25:42 +01:00
huangdarwin
97272c139c Effects: Output to texture without surface in VFP.
Allow the VideoFrameProcessor to output to a texture without an output surface.

Tested by updating texture output tests to no longer output to a surface.

PiperOrigin-RevId: 527244605
2023-04-26 15:51:06 +01:00
tonihei
889f435a49 Update dependencies to latest versions
The only dependencies that are not updated are the ones that need
to be kept in sync with other system (like Android source tree) or
would require a Kotlin dependency in common or exoplayer modules.

As a side effect, some demo apps now need a Kotlin config and some
additional modules require desugaring/multidex logic. To simplify
the setup, the desugaring and multidex steps are added to the common
config.

PiperOrigin-RevId: 527243950
2023-04-26 15:49:57 +01:00
huangdarwin
d8cf4449a4 Transformer: Reduce scope of createSupportedTransformationRequest
This was only used in EncoderWrapper, so move createSupportedTransformationRequest
from VideoSamplePipeline to EncoderWrapper to reduce its scope.

PiperOrigin-RevId: 527226578
2023-04-26 15:48:50 +01:00
tofunmi
8612d2820d Update image transcoding pipeline to signal input COLOR_TRANSFER_SRGB
The production code changes are in transformer, but the tests in effect have also been updated to confirm the is no color regression `inputColorInfo.colorTransfer=C.COLOR_TRANSFER_SRGB`

PiperOrigin-RevId: 526950435
2023-04-26 15:46:21 +01:00
tofunmi
42c9c28daf Update VideoSamplePipeline to take account of decoder colors
PiperOrigin-RevId: 526940261
2023-04-26 15:44:52 +01:00
huangdarwin
324115f6cf Effects: Allow BT2020 colors with SDR transfers
Previously, I assumed that SDR contents must be BT709, and HDR contents must be
BT2020. Turns out BT2020 is just wide-gamut, and SDR contents / transfers may be
represented in BT2020 color spaces.

Relax the check, so that we don't throw when valid BT2020 SMPTE 170M contents
are input into effects.

PiperOrigin-RevId: 526668347
2023-04-26 15:41:33 +01:00
sheenachhabra
b11dd106ae Replace MediaFormat with Format class in muxer module
PiperOrigin-RevId: 526655859
2023-04-26 15:40:40 +01:00
tofunmi
f1d9a0a819 Add nullness-compat-qual dependency for android tests
PiperOrigin-RevId: 526652129
2023-04-26 15:38:22 +01:00
ibaker
2d968a5d97 Rollback of 8d17faea33
*** Original commit ***

Rollback of d7983f9485

*** Original commit ***

Bump Guava version to 31.1

***

***

PiperOrigin-RevId: 526601244
2023-04-26 15:36:00 +01:00
claincly
fcec5a29c1 Fix not ignoring invalid color info.
PiperOrigin-RevId: 526595567
2023-04-26 15:34:52 +01:00
huangdarwin
de32536e20 HDR: Remove exportUnexpectedColorInfo test.
PiperOrigin-RevId: 526086973
2023-04-24 11:39:09 +01:00
tofunmi
0f8fa232e7 Change isInputTextureExternal boolean parameter to inputType intDef
PiperOrigin-RevId: 525690361
2023-04-24 11:21:41 +01:00
huangdarwin
3bc0172188 Remove extra param from createForVideoFrameProcessingException
This arg was always the same value, so it can be simplified.

PiperOrigin-RevId: 525677735
2023-04-24 11:20:32 +01:00
tofunmi
316a2ce0ee Remove deprecated constructor in DefaultVideoFrameProcessor.Factory
Remove deprecated factory constructor and replace all call instances of it.

PiperOrigin-RevId: 525476062
2023-04-24 11:19:49 +01:00
tofunmi
9c3d30791a Move glObjectsProvider to DefaultVideoFrameProcessor.Factory.Builder
Removes glObjectsProvider from `VideoFrameProcessor` and `Effects`. Apps will set the glObjectsProvider on the DefaultVideoFrameProcessor.Factory.Builder when providing a custom DefaultVideoFrameProcessor.Factory, rather than in `Effects`.

PiperOrigin-RevId: 525169059
2023-04-18 18:12:14 +01:00
andrewlewis
990f7ef5b5 Select codecs without util methods
`findDecoder/EncoderForFormat` seem to be more restrictive than querying
`MediaCodecList` as done for playback, and it's not clear that the cases where
no codec is found correspond to actual cases that would fail given that it
seems this returns an empty string for many cases in production.

Switch to using `MediaCodecUtil` and `EncoderUtil` for querying codecs instead.

PiperOrigin-RevId: 525078321
2023-04-18 18:08:18 +01:00
kimvde
23b1cab56c Fix video timestamp 0 set to large value in some cases
PiperOrigin-RevId: 525064269
2023-04-18 18:07:19 +01:00
Googler
ffa969f97c Expose MssimCaluclator for use in Duetdev tests
Child: <unknown commit>

PiperOrigin-RevId: 524876651
2023-04-18 18:06:01 +01:00