1607 Commits

Author SHA1 Message Date
kimvde
2df50f209e CompositionPlayerTest: increase timeout for emulator tests
Some tests are sometimes timing out because the emulator is too slow.

PiperOrigin-RevId: 713671851
2025-01-09 07:56:55 -08:00
tonihei
b9d12837b4 Use live providers in CompositionPlayer to remove workaround
When CompositionPlayer added repeat modes in d0afb96c40, it
changed the logic in SimpleBasePlayer to workaround the specific
scenario in CompositionPlayer that didn't seem to work correctly.
The workaround is not really generically applicable though. The
source of the original problem was that the position values of
all state objects were connected to the "live" source of the
current player so that the repetition case couldn't be detected
(the position before and after the seek looked the same). The
solution is to use the newly added LivePositionSuppliers and
disconnect them before seeking.

PiperOrigin-RevId: 713659638
2025-01-09 07:10:14 -08:00
Googler
8a709a7d76 Remove setPendingVideoEffects from VideoSink.
VideoSink#onInputStreamChanged(int, Format, List<Effect>) should now be used to set video effects on a new input stream.

PiperOrigin-RevId: 713627389
2025-01-09 04:48:21 -08:00
kimvde
04ff17d939 CompositionPlayerSeekTest: skip some tests on API 31 emulator
The audio decoder is sometimes failing with error "previous call to
queue exceeded timeout - Codec reported err 0x80000000, actionCode 0,
while in state 6/STARTED". This is probably because the emulator is too
slow.

PiperOrigin-RevId: 713564117
2025-01-09 00:24:31 -08:00
kimvde
d9776e74c8 PlayerListener: do not swallow exceptions thrown before waiting
If an error occurred in the player before the test was calling one of
the waitUntilSomething() methods, a timeout exception was thrown instead
of the actual error.

PiperOrigin-RevId: 713292292
2025-01-08 08:03:46 -08:00
kimvde
bb3b85a359 CompositionPlayerSeekTest: don't hide exception while waiting on latch
If a playback exception was thrown before or during awaiting the
frameCountBeforeBlockLatch, a TimeoutException was thrown instead of the
playback exception, hiding the actual cause of the failing test.

PiperOrigin-RevId: 713279627
2025-01-08 07:10:09 -08:00
kimvde
30038079c4 Split CompositionPlayerSeekTest.playSequenceAndGetTimestampsUs
This method is already quite complex and I will need to add more
complexity, so split it into multiple smaller methods.

This CL is a refactoring. There are no functional changes.

PiperOrigin-RevId: 713266816
2025-01-08 06:18:48 -08:00
kimvde
48e3eaa623 CompositionPlayerSeekTest: increase test timeout for emulators
Some tests are sometimes timing out because the emulator is too slow.

PiperOrigin-RevId: 713259939
2025-01-08 05:48:18 -08:00
shahddaghash
1bea11637b Add setting final progress at export error or cancellation
The change includes setting the final progress percentage when the export completes with error or is cancelled. The value for the progress is collected by calling `Transformer.getProgress()` before `transformerInternal` is set to null.

PiperOrigin-RevId: 713227568
2025-01-08 03:24:18 -08:00
shahddaghash
d894217a4e Fix typo for precreateEditingEndedEventBuilder method
PiperOrigin-RevId: 712969879
2025-01-07 10:56:15 -08:00
sheenachhabra
bcae7abde9 Remove TrackToken and replace with int trackId
This is to make API simpler and to have parity with `MediaMuxer`

PiperOrigin-RevId: 712954669
2025-01-07 10:16:31 -08:00
shahddaghash
8dca430e42 Add setting final progress when export success for metrics
Set the progress to 100% for a successful export operation.

PiperOrigin-RevId: 712946218
2025-01-07 09:52:43 -08:00
andrewlewis
e11aabb794 Allow RawAssetLoader media to have unset duration
This will be useful for recording use cases, including screen recording with
audio queued via `RawAssetLoader`, where the duration is unknown.

PiperOrigin-RevId: 712895577
2025-01-07 06:59:21 -08:00
andrewlewis
9ac91aaadd Return concrete SurfaceAssetLoader from factory
If the caller is another asset loader (or another asset loader factory) that is
delegating to a `SurfaceAssetLoader`, it may want to call methods on the
`SurfaceAssetLoader`. Return the concrete type so that a cast isn't necessary
at the call site.

PiperOrigin-RevId: 712881130
2025-01-07 06:02:24 -08:00
shahddaghash
0074a97333 Remove unnecessary released condition in getProgress
Removed the check for `released` flag to be able to pass the actual progress when the export completes.

Also, removed resetting the `progressState` and `progressValue` to be able to get the actual progress. This block is also not necessary to have at this point.

PiperOrigin-RevId: 712851526
2025-01-07 03:53:06 -08:00
Googler
b2aa8d6d21 Add sample rate fallback to AudioSampleExporter
After this change, if the sample rate supported by the encoder differs from the requested sample rate and enableFallback is true, the AudioSampleExporter will convert audio to a sample rate supported by the encoder. This fixes a bug where the audio track is distorted when an unsupported sample rate is requested.

PiperOrigin-RevId: 712822358
2025-01-07 01:52:57 -08:00
shahddaghash
115f09dd6d Report ErrorCode and elapsed time metrics for Transformer
Added reporting 2 values from Transformer:
- elapsed time since creation of the `editingSession` in milliseconds.
- the `ErrorCode` for export errors. The error codes were mapped from `@ExportException.ErrorCode` int to `@EditingEndedEvent.ErrorCode`.

PiperOrigin-RevId: 712472849
2025-01-06 03:55:32 -08:00
sheenachhabra
750baf2c05 Rollback of 9cc4532f84
PiperOrigin-RevId: 711804077
2025-01-03 11:18:03 -08:00
shahddaghash
cd5d5bde27 Implement basic Metrics Collector in Transformer
Added a new `EditingMediaMetrics` class that interacts with the platform's `MediaMetricsManager` through an `EditingSession` created when export starts.

Currently, only the `finalState` of the export event is reported to the `EditingSession`. Future changes will collect additional metrics based on the export operation's output.

PiperOrigin-RevId: 711721801
2025-01-03 05:27:20 -08:00
sheenachhabra
01c51d8475 Allow dolby vision transcode test to use alternative output mime type
On many devices (for example pixel 7) a dolby vision file can be
decoded (either using dolby vision decoder or H265 decoder)
but they can't be re-encoded to dolby vision.
Allow test to encode to H265.

The test now passes on pixel 7. It was getting skipped earlier.

PiperOrigin-RevId: 711383794
2025-01-02 04:15:27 -08:00
sheenachhabra
9cc4532f84 Use same decoder selection logic is test as in DefaultDecoderFactory
The decoder selection logic in DefaultDecoderFactory is more mature
and can find alternative decoders.

PiperOrigin-RevId: 711378121
2025-01-02 03:49:52 -08:00
sheenachhabra
969c50d60f Fix HDR editing support check in dolby vision test
A device (for example pixel 7) might support HDR encoding
but with H265 sample mime type but the existing condition made
assumption false on such devices.

The test still does not pass on pixel 7 because the format
matching condition is incorrect. It will be fixed in a separate CL.

PiperOrigin-RevId: 711368798
2025-01-02 03:00:08 -08:00
claincly
515de89973 Support MediaCodec tonemapping in CompositionPlayer
PiperOrigin-RevId: 708995517
2024-12-23 02:06:08 -08:00
claincly
8188f7a865 Repect Composition's HDR mode setting using OpenGL
Subsequent CLs will add support for other HDR modes

PiperOrigin-RevId: 708340216
2024-12-20 09:11:00 -08:00
Googler
c12b1768a6 Add sample rate fallback to DefaultEncoderFactory
After this change if a sample rate is requested that is not supported by the available encoders and enableFallback is true, DefaultEncoderFactory will fall back to the encoder with the closest matching sample rate.

In the case when an encoding profile is included in requestedAudioEncoderSettings, an encoder matching that profile will continue to be preferenced.

PiperOrigin-RevId: 708295869
2024-12-20 05:50:25 -08:00
shahddaghash
915130eb00 Add usePlatformDiagnostics in Transformer.
Added a new `usePlatformDiagnostics` in Transformer. This parameter enables/disables forwarding editing events and performance data to the platform.

This is pre-work for metrics support in Transformer. In the following CLs, metrics collection and forwarding will be implemented.

PiperOrigin-RevId: 707930368
2024-12-19 09:03:19 -08:00
sheenachhabra
44aa87c832 Extract MuxerException into a separate file
This is to avoid very long name as the FQN earlier
contained muxer 3 times

PiperOrigin-RevId: 707591813
2024-12-18 10:21:51 -08:00
dancho
72672b7f87 Implement audioNeedsEncoding based on requested settings
Do not use ForceEncodeFactory in tests

PiperOrigin-RevId: 705835243
2024-12-13 04:23:35 -08:00
dancho
522883bf16 Less granular skipping for HE-AAC support
PiperOrigin-RevId: 705822449
2024-12-13 03:28:51 -08:00
kimvde
c636d9181b Delete ExoPlayerEffectPlaybackSeekTest
This test was testing a scenario that never happens in production.
Indeed, in this test, a frame was registered to the
ExternalTextureManager but never rendered on the decoder ouput surface.
This never happens in production because both actions are normally
performed in
PlaybackVideoGraphWrapper.InputVideoSink.handleInputFrame().

This test was also failing on API 31 emulator. As the frame was never
rendered, the timeout to release all registered frames in
ExternalTextureManager was always exceeded. This sometimes caused the
decoder to fail on this emulator because there was no interaction with
the decoder for a long time.

PiperOrigin-RevId: 705490910
2024-12-12 07:05:34 -08:00
microkatz
55d9626b5a Format with google-java-format 2024-12-11 14:56:25 +00:00
Gilles Khouzam
f149cb28a3 Add MediaCodecInfo to MediaCodecRenderer::onReadyToInitializeCodec 2024-12-11 14:56:25 +00:00
dancho
f55e0f3a3f VideoFrameProcessorTestRunner add support for portrait video
Fixes test failures due to differences test util bug that introduced
a crop in external texture sampling

PiperOrigin-RevId: 705061267
2024-12-11 04:45:12 -08:00
kimvde
1a7da45dd9 Add a TestTransformerBuilder
This is inspired from TestExoPlayerBuilder.

Adding this class has a few advantages:
- It makes testing easier for apps by configuring Transformer for unit
  tests (for example, by setting the clock).
- It removes the Transformer setters that apps should not use for unit
  testing (for example, the video MIME type should not be set because
  it would cause Transformer to transcode.
- It allows us to add additional setters in the future, for example to
  build a Transformer that always fails.

PiperOrigin-RevId: 704181927
2024-12-09 01:06:51 -08:00
dancho
1d2ffcb165 Frame extraction support changing MediaItem
Add FrameExtraction.setMediaItem() method
Prefer non-vendor software codecs by default because
vendor codecs sometimes fail when we attempt reuse.

PiperOrigin-RevId: 703486235
2024-12-06 06:57:21 -08:00
shahddaghash
862791f837 Fixes after removing deprecated methods from Transformer.Builder
After removing `Transformer.Builder.setFlattenForSlowMotion()`, there is no need to keep `flattenForSlowMotion` in Transformer since it's now set in `EditedMediaItem`.
The change also includes making `audioProcessors` and `videoEffects` attributes final.

PiperOrigin-RevId: 703076023
2024-12-05 05:14:59 -08:00
kimvde
f1a0e4b0b7 Fix timeouts in CompositionPlaybackTest
Some of the tests were timing out because the emulator was too slow.

PiperOrigin-RevId: 703056616
2024-12-05 03:54:49 -08:00
dancho
6193f7c38f Generate static HDR metadata when using DefaultEncoderFactory
* Select an encoder that supports HDR editing.
* Set KEY_PROFILE to an HDR10 option
* Use DecodeOneFrameUtil test util to return the MediaCodec format,
  which includes HDR_STATIC_INFO

PiperOrigin-RevId: 702752639
2024-12-04 09:31:13 -08:00
dancho
e8f60d4fec Remove earlier frame extraction prototypes
Earlier this year, we explored frame extraction built on top of Transformer
or ImageReader.

We decided against these approaches because random-access of frames
is a key requirement. And ImageReader behaviour is unreliable.

PiperOrigin-RevId: 702303490
2024-12-03 05:32:09 -08:00
dancho
7821e7702b Frame extractor: explain color accuracy issues
This is only a javadoc change, setting expectations.

PiperOrigin-RevId: 702295210
2024-12-03 04:56:38 -08:00
kimvde
0037388660 Simplify VideoSink.handleInputFrame
This method was taking positionUs and elapsedRealtimeUs parameters and
throwing a VideoSinkException because it was calling render() to make
room for a new input frame. Instead of doing that, this CL makes sure
render() is called before  handleInputFrame() in the renderer (even
though it may not make a significant difference).

PiperOrigin-RevId: 702273587
2024-12-03 03:29:30 -08:00
claincly
675c1d898a Move interface locations
PiperOrigin-RevId: 702096389
2024-12-02 15:03:37 -08:00
claincly
ecdf7c5df7 Propagate sourceEnded to AudioGraphInputAudioSink
PiperOrigin-RevId: 702076933
2024-12-02 13:59:15 -08:00
claincly
8908d82cac Signal end of stream to VideoFrameProcessor in previewing
This will be used by VideoCompositor

PiperOrigin-RevId: 702031907
2024-12-02 11:30:20 -08:00
dancho
b2ba5da09b Implement cancellation of FrameExtractor.getFrame
Previously cancelling one FrameExtractor.getFrame
ListenableFuture caused the following requests to fail.
Implement ability to cancel Futures, and correctly issue
next player seek commands after all past requests complete.

Removes uses of SettableFuture, and replaces them with
CallbackToFutureAdapter.
Adds a dependency on androidx.concurrent:concurrent-futures

PiperOrigin-RevId: 701941403
2024-12-02 06:09:33 -08:00
Googler
19b276d6a7 Allow track selection parameters to be set in ExoPlayerAssetLoader.
PiperOrigin-RevId: 701926949
2024-12-02 05:04:26 -08:00
dancho
d214e90ce4 Fix Frame Extractor getting stuck
Frame Extractor was getting stuck with SeekParameters.CLOSEST_SYNC.
onPositionDiscontinuity callback was sometimes being called with a
non-adjusted new position.

Fix this by monitoring player state ready.
For the player to become ready, we have to override renderer isReady.

PiperOrigin-RevId: 701924752
2024-12-02 04:56:26 -08:00
kimvde
f3f4646296 Add VideoSink.signalEndOfCurrentInputStream()
The isLastBuffer flag passed to MediaCodecRenderer.processOutputBuffer()
is unreliable. It is true for the last frames (plural) of a video
stream, which makes it possible for the video renderer to end before all
the frames have been rendered to the output surface.

PiperOrigin-RevId: 700941685
2024-11-28 01:23:55 -08:00
kimvde
dfe3c90f9a Fix CompositionPlayerTest.imagePreview_imagePlaysForSetDuration
PiperOrigin-RevId: 700931124
2024-11-28 00:36:46 -08:00
dancho
6e29b96337 Add support for extracting HDR frames as HLG Bitmaps
* Add GlUtils to create RGB10_A2 textures.
* Add OETF conversion to FrameReadingGlShaderProgram.
* Add FrameExtractor configuration API to request HDR output: API 34+

PiperOrigin-RevId: 700738895
2024-11-27 10:18:57 -08:00