833 Commits

Author SHA1 Message Date
tofunmi
2af5752785 Revert AndroidTestUtil.canDecode to use EncoderUtil.findCodecForFormat
208eefc0fd introduced using `DefaultDecoderFactory.getDecoderInfo(format) != null` caused certain tests not to be skipped when they were expected to be, creating more mh failures.

PiperOrigin-RevId: 537820370
2023-06-06 17:57:47 +00:00
andrewlewis
74478f2478 Tidy color info checking tests
ExoPlayer extractors (backing `MetadataRetriever`) now parse the color format
from the bitstream so using `MetadataRetriever` should be an equivalent but
more lightweight way to verify the color info.

Also remove try/catch blocks in test code calling into these methods, and add
skipping based on decoder capabilities in the cases where it was missing.

PiperOrigin-RevId: 537789483
2023-06-05 09:51:01 +00:00
ibaker
7e6dae7b31 Remove exoplayer2-only javadoc machinery from media3
This introduces whitespace-only changes in exoplayer2 due to the way
copybara stripping works.

PiperOrigin-RevId: 537296933
2023-06-05 09:47:58 +00:00
huangdarwin
c52130a212 Test: Add special effects for forcing transcode.
Previously, we would apply a general effect to signal wanting to transcode.

PiperOrigin-RevId: 537034455
2023-06-02 09:16:52 +00:00
kimvde
108000834b Add TransformationRequest toString method
PiperOrigin-RevId: 536727079
2023-05-31 18:00:12 +00:00
claincly
208eefc0fd Fix codec's MIME type is not used
In some cases the codec selected for decoding has a different MIME type than
the media. In thoses cases Transformer continued to use the media's MIME type
and that caused codec configuration failures.

Removed `EncoderUtil.findCodecForFormat()` as we stopped using the method it
uses for finding a codec. Plus, the method is only used in the test.

See also `MediaCodecUtil.getALternativeCodecMimeType()`.

PiperOrigin-RevId: 536683663
2023-05-31 17:57:40 +00:00
sheenachhabra
99d2cf6713 Add only supported MdtaMetadataEntry
PiperOrigin-RevId: 536351494
2023-05-30 14:24:06 +00:00
tofunmi
94d29f35fc Image transcoding: Add support for bmp image format.
With this change we will now support loading bitmaps from all the formats documented [here](https://developer.android.com/guide/topics/media/media-formats#image-formats) except for gifs (because they are animated). Java doc is added to express this.

PiperOrigin-RevId: 535610152
2023-05-26 15:16:17 +00:00
kimvde
f4d1a6c453 Transmux video if rotation is only effect applied
PiperOrigin-RevId: 535554628
2023-05-26 15:14:52 +00:00
tofunmi
a19e07c4d2 Move OnInputFrameProcessedListener into it's own file
PiperOrigin-RevId: 535183521
2023-05-25 15:09:36 +00:00
sheenachhabra
a944ffecb9 Add support for adding capture FPS via transformer
PiperOrigin-RevId: 534814892
2023-05-24 16:11:27 +01:00
sheenachhabra
71facd825e Add support for adding XMP data via transformer
PiperOrigin-RevId: 534801202
2023-05-24 16:09:43 +01:00
huangdarwin
66554b9b68 Codec: Reduce limit for max decoder pending output frames.
Tentative/experimental value to reduce codec timeouts. We will reconsider using a larger limit after seeing whether this really does reduce error rate.

PiperOrigin-RevId: 534491615
2023-05-24 16:05:00 +01:00
sheenachhabra
7c477589e5 Add support for updating Metadata entries via InAppMuxer
Mp4Muxer already supports writing Mp4LocationData so added that
as supported Metadata entry.
Support for more Metadata entries will be added in upcoming CLs.

PiperOrigin-RevId: 534473866
2023-05-24 16:02:48 +01:00
sheenachhabra
a9e3f5def4 Write metadata to Mp4Muxer in the release() method
Earlier metadata was written multiple times as it came.
With new changes, all the distinct metadata entries will
get collected and will be written at once in the end.

PiperOrigin-RevId: 534088401
2023-05-24 15:43:46 +01:00
huangdarwin
a6897aedaa Effect: Add GlTextureInfo release() and accessor methods.
This allows us to disallow access after release.

PiperOrigin-RevId: 534046475
2023-05-24 15:42:47 +01:00
andrewlewis
b0418f1a2a Use Ascii for conversion to lower case
PiperOrigin-RevId: 534016337
2023-05-24 15:34:40 +01:00
ibaker
d5f9cf4f19 Add singleVariant publishing config to all published gradle files
Docs:
* https://developer.android.com/build/publish-library/configure-pub-variants#single-pub-var
* https://developer.android.com/reference/tools/gradle-api/8.0/com/android/build/api/dsl/PublishingOptions

This resolves the following warning from Android Gradle Plugin 7.2.2 and
helps unblock the upgrade to AGP 8.0.1:

> Software Components will not be created automatically for Maven
> publishing from Android Gradle Plugin 8.0. To opt-in to the future
> behavior, set the Gradle property
> `android.disableAutomaticComponentCreation=true` in the
> `gradle.properties` file or use the new publishing DSL.
>
> Affected Modules: `lib-cast`, `lib-common`, `lib-container`,
> `lib-database`, `lib-datasource`, `lib-datasource-cronet`,
> `lib-datasource-okhttp`, `lib-datasource-rtmp`, `lib-decoder`,
> `lib-effect`, `lib-exoplayer`, `lib-exoplayer-all (legacy)`,
> `lib-exoplayer-dash`, `lib-exoplayer-hls`, `lib-exoplayer-ima`,
> `lib-exoplayer-rtsp`, `lib-exoplayer-smoothstreaming`,
> `lib-exoplayer-workmanager`, `lib-extractor`, `lib-media2 (legacy)`,
> `lib-mediasession (legacy)`, `lib-muxer`, `lib-session`,
> `lib-transformer`, `lib-ui`, `lib-ui-leanback`, `test-utils`,
> `test-utils-robolectric`

Issue: androidx/media#409
PiperOrigin-RevId: 533464133
2023-05-19 17:22:21 +01:00
ibaker
b3db85133a Add android.namespace to all build.gradle files
This is a pre-requisite for the Android Studio upgrade assistant to
upgrade from AGP 7.2.2 to 8.0.1, otherwise it fails and complains
this is missing.

Issue: androidx/media#409
PiperOrigin-RevId: 533463246
2023-05-19 17:19:15 +01:00
ibaker
0f8ddd5d1a Suffix package in test AndroidManifest.xml files with .test
This is blocking the upgrade to AGP 8.0.1. The Android Studio Upgrade
Assistant refuses to run with the error:
> Modules have the same package for their `main` and `androidTest`
> artifacts
>
> The package specifications in `AndroidManifest.xml` files define the
> same package for the `main` and `androidTest` artifacts, in the
> following modules: `effect`, `muxer`, `transformer`,
`test_exoplayer_abr`, `test_exoplayer_performance`, and `container`.

This change also adds the `.test` suffix to `test/AndroidManifest.xml`
files where it's missing, because although it's not mentioned in the
Upgrade Assistant error it does cause problems later.

```
> Incorrect package="androidx.media3.database" found in source AndroidManifest.xml: libraries/database/src/test/AndroidManifest.xml.
  Setting the namespace via the package attribute in the source AndroidManifest.xml is no longer supported.
  Recommendation: remove package="androidx.media3.database" from the source AndroidManifest.xml: libraries/database/src/test/AndroidManifest.xml.
```

Issue: androidx/media#409
PiperOrigin-RevId: 533460180
2023-05-19 17:18:22 +01:00
huangdarwin
25fa2df2de Effect: Use callback to release texture
This allows us to avoid needing a reference to the VideoFrameProcessor, which
can be especially difficult if an App only has a reference to the
VideoFrameProcessor.Factory it passes into Transformer/ExoPlayer.

PiperOrigin-RevId: 533205983
2023-05-19 17:10:53 +01:00
tofunmi
5fe10d7652 HDR texture asset loading
PiperOrigin-RevId: 532846248
2023-05-18 12:21:38 +00:00
sheenachhabra
410840c9e1 Add default constructor for InAppMuxer.Factory
PiperOrigin-RevId: 532838813
2023-05-18 12:18:43 +00:00
huangdarwin
83190a0fe9 HDR: Blocklist Galaxy Z Fold 4 for HLG tone mapping.
This device failed on HdrEditingTest's exportAndTranscode_hlg10File_whenHdrEditingUnsupported_toneMapsOrThrows
before this CL, and succeeds on that test after this CL.

PiperOrigin-RevId: 532796897
2023-05-18 12:15:47 +00:00
sheenachhabra
7b62b33127 Move Mp4LocationData from extractor module to container module
This class will be shared between extractor and muxer module.

PiperOrigin-RevId: 532784415
2023-05-17 16:37:57 +01:00
sheenachhabra
e0d6f67dd9 Move NAL unit related methods from common to container module
PiperOrigin-RevId: 532750099
2023-05-17 16:32:46 +01:00
tofunmi
88642587ac Update mixed input test to include images of different aspect ratios
Makes this test a little more thorough.

PiperOrigin-RevId: 532386515
2023-05-17 08:35:36 +01:00
huangdarwin
62afbe87bb Test: Suppress SSIM timeouts on galaxy fold/flip.
This is a failure only in SSIM, so it seems unlikely we'll prioritize this over
other work or bugs soon. Suppress test failures to reduce triage burden.

PiperOrigin-RevId: 532200729
2023-05-16 09:59:36 +01:00
huangdarwin
07ec1eaa48 Effect: Multiple Texture output
Allow the VideoFrameProcessor to output multiple textures at a time, so that
lifetime of textures is up to the consumer calling VFP.releaseOutputFrame.

The FinalShaderProgramWrapper also has a new maxCapacity limit added, to ensure
the a reasonable amount of textures is used and avoid using up memory.

PiperOrigin-RevId: 532094256
2023-05-16 09:49:52 +01:00
kimvde
6dfb387117 Refactor SequenceAssetLoader release
In the past, the SequenceAssetLoader was released in TransformerInternal
when the export ended.

fc539da061 was made to release the SequenceAssetLoader earlier, when
loading ended. This was causing player release timeouts because the last
AssetLoader in the sequence was released before the SamplePipelines (see
0b40bc37ab for more information).

The code that was releasing the SequenceAssetLoader was first commented
out because we didn't have an understanding of what was happening.

This change removes the early SequenceAssetLoader release all together.
It doesn't have any effect as this code was already commented out.

PiperOrigin-RevId: 532065673
2023-05-16 09:43:36 +01:00
huangdarwin
2736b11845 Transformer: Add latest input format to DebugTraceUtil
This can provide more information to help debug muxer errors

PiperOrigin-RevId: 531521974
2023-05-15 10:39:57 +01:00
sheenachhabra
867355fdc5 Rollback of 925aa34e13
*** Original commit ***

Rollback of 65d5132f76

*** Original commit ***

Create InAppMuxer in transformer

To use the InAppMuxer, the client needs to pass InAppMuxer Factory.

***

***

PiperOrigin-RevId: 531470081
2023-05-12 13:20:31 +00:00
huangdarwin
5313763eb3 Test: Filter test more precisely, with model instead of manufacturer
PiperOrigin-RevId: 531233480
2023-05-12 10:10:15 +01:00
huangdarwin
674885cfa5 Test: Suppress failing 8k24 SSIM bug.
PiperOrigin-RevId: 531193242
2023-05-12 10:07:55 +01:00
kimvde
203450d244 Small fixes in TextureAssetLoader
The texture input tests in TransformerEndToEndTest were not passing on
Pixel 7. Implemented a fix and fixed other minor threading issues I
spotted while looking at the code.

PiperOrigin-RevId: 531141659
2023-05-11 10:07:43 +00:00
kimvde
e8072ca2c7 Move VideoFrameProcessorFactory setter to Transformer
PiperOrigin-RevId: 531123743
2023-05-11 10:06:21 +00:00
huangdarwin
82ede47398 media3: Capitalize "MIME" for MIME types.
Find and replace "mime" to "MIME", where appropriate, throughout media3.

PiperOrigin-RevId: 531122121
2023-05-11 10:05:01 +00:00
Googler
925aa34e13 Rollback of 65d5132f76
*** Original commit ***

Create InAppMuxer in transformer

To use the InAppMuxer, the client needs to pass InAppMuxer Factory.

***

PiperOrigin-RevId: 531056436
2023-05-11 10:03:35 +00:00
tofunmi
7fea435b7e Transformer: Support Texture asset loading
PiperOrigin-RevId: 530888319
2023-05-11 10:01:00 +00:00
Googler
ecaf219d74 Adding mime type checking to Transformer to identify images.
PiperOrigin-RevId: 530711523
2023-05-11 09:58:15 +00:00
sheenachhabra
65d5132f76 Create InAppMuxer in transformer
To use the InAppMuxer, the client needs to pass InAppMuxer Factory.

PiperOrigin-RevId: 530684007
2023-05-11 09:56:46 +00:00
kimvde
dc4c6daf0b Fix HDR tone mapping stuck on SM device
Tone-mapping an HDR video with MediaCodec on sm-s908u1 was timing out.
The reason for that is that the decoder was dropping frames, and the
ExternalTextureManager was therefore never propagating the end-of-stream
signal.

There was already a workaround for a similar issue but restricted to
sm-f936b. Removed the model check as the bug is probably present on more
devices.

PiperOrigin-RevId: 530639437
2023-05-11 09:53:25 +00:00
kimvde
1105f194ca Avoid swallowing exceptions when switching AssetLoader
Exceptions thrown when switching AssetLoader were not reported through
the error listener. Therefore, the resources were not released and the
export was not ending.

PiperOrigin-RevId: 530626300
2023-05-11 09:50:43 +00:00
tofunmi
7875131e2a Effect: Support input via texture ID
PiperOrigin-RevId: 530624195
2023-05-11 09:49:26 +00:00
huangdarwin
2db2de5993 Transformer: Add codec support for Dolby Vision HDR video
Allow use of H265/H264 codecs for Dolby Vision video.

Also, reflow ExoPlayer code to use this new utility class

PiperOrigin-RevId: 530619388
2023-05-11 09:47:52 +00:00
kimvde
129a6e0cc6 Fix encoder expecting HDR while tone-mapping requested
If tone-mapping was requested for a device supporting HDR encoding,
isToneMapped was false in VideoSamplePipeline.EncoderWrapper. This was
causing the encoder to expect HDR.

Also did some renamings to improve readability

PiperOrigin-RevId: 530584010
2023-05-11 09:45:09 +00:00
tofunmi
b1ebd0e2a6 Change order of operations in videoSamplePipeline
similar error to 9baa6f6be5L `registerInputStream` sets the textureManager, so reordering makes sure the textureManager is set before you set the frame info. This is important for texture input, where the frame info provides the width and height.

PiperOrigin-RevId: 530579584
2023-05-11 09:43:40 +00:00
tofunmi
477994461d Image transcoding: Add support for heic/heif image formats
PiperOrigin-RevId: 530578549
2023-05-11 09:42:12 +00:00
andrewlewis
f0ff57882a Remove stray stuck debug logging line
PiperOrigin-RevId: 530314222
2023-05-11 09:39:17 +00:00
kimvde
30957df635 Remove B-frame setting from DefaultEncoderFactory
This setting is buggy on some devices (for example sm-n960f)

PiperOrigin-RevId: 529712770
2023-05-05 16:54:27 +00:00