9874 Commits

Author SHA1 Message Date
kimvde
b88025cc3d Fix stuck muxer
Unstuck the muxer if the next timestamp in the track with the minimum
timestamp is larger than this minimum timestamp plus
MAX_TRACK_WRITE_AHEAD_US.

PiperOrigin-RevId: 510977088
2023-02-24 09:53:06 +00:00
huangdarwin
c79eb7466a Effect: Add FloatRange to public GlEffect impl interfaces.
Using these annotations/checkers should hopefully make it marginally harder to use the wrong input values in the API.

PiperOrigin-RevId: 510966941
2023-02-24 09:49:37 +00:00
christosts
cbb6878f9f Skip rendering multiple frames on the same vsync
When rendering frames at a rate higher than the screen refresh rate,
e.g. playing at 8x, the player is releasing multiple frames at the same
release time (nanos) which are then dropped by the platform. The output
buffers are available later and as a result MediaCodec cannot keep up
decoding fast enough.

This change skips releasing multiple video frames on the same vsync
period and proactivelly drops the frame. The frame is counted as skipped
rather than dropped to differentiate with frames dropped due to slow
decoding.

PiperOrigin-RevId: 510964976
2023-02-24 09:46:06 +00:00
tonihei
829b49d724 Reduce number of calls to AudioTrack.getPlaybackHeadPosition
This call may cause performance overhead in some situations,
for example if the AudioTrack needs to query an offload DSP
for the current position. We don't need to check this multiple
times per doSomeWork iteration as the value is unlikely to
change in any meaningful way.

PiperOrigin-RevId: 510957116
2023-02-24 09:42:39 +00:00
huangdarwin
6a91b2df51 HDR: Remove comment explaining force sdr behavior with assetloaders.
Before, if the upstream AssetLoader provides HDR to the VideoSamplePipeline when
HDR_MODE_EXPERIMENTAL_FORCE_INTERPRET_HDR_AS_SDR is requested, the
VideoSamplePipeline would attempt to tell the AssetLoader to output SDR, which
could be accomplished via MediaCodec tone-mapping in the AssetLoader.

However, this makes an assumption of the AssetLoader implementation, and
AssetLoaders may not all implement support for decoder tone-mapping. Remove javadoc
attempting to explain how AssetLoaders (ex. custom ones) could behave.

PiperOrigin-RevId: 510956820
2023-02-24 09:39:08 +00:00
samrobinson
363b85226e Pass AudioFormat to ASP#computeNextEncoderInputBufferTimeUs.
This method uses sampleRate, channelCount and pcmEncoding, so passing
AudioFormat is easier.

This will lead into a future change that builds the
encoderInputAudioFormat from encoder.getConfigurationFormat()

PiperOrigin-RevId: 510956177
2023-02-24 09:35:35 +00:00
huangdarwin
ae37d33b18 Effect: Rename to DefaultShaderProgram.
Rename:
* MatrixShaderProgram to DefaultShaderProgram, and
* FinalMatrixShaderProgramWrapper to FinalShaderProgramWrapper.
PiperOrigin-RevId: 510498547
2023-02-24 09:32:08 +00:00
huangdarwin
540f5d681d Effect: Rename to ScaleAndRotateTransformation
Rename ScaleToFitTransformation to ScaleAndRotateTransformation.

This better represents the operations that can be accomplished using this
effect. The name was originally named ScaleToFit* because it's not obvious how
to scale to fit using OpenGL, and this effect handled the scaling to fit in a way that no other MatrixTransformations did.

However, it's hard to discover how to rotate when skimming names of effects, so
it's probably more useful to convey that this effect rotates, than that it
scales to fit.

PiperOrigin-RevId: 510480078
2023-02-24 09:28:37 +00:00
huangdarwin
84fe683ca3 HDR: Add colorinfo to exception logs.
Format.toString unfortunately doesn't log colorInfo, and as Format holds a very
large set of values, it's unclear that it should. ColorInfo is useful for codec
exceptions though, so log this in ExportException.createForCodec.

PiperOrigin-RevId: 510475520
2023-02-24 09:25:07 +00:00
tofunmi
5e93616ad2 fix nits and enhance DefaultVideoFrameProcessorImageFrameOutputTest
resolving comments from 8aa1638fe4.

PiperOrigin-RevId: 510457401
2023-02-24 09:21:39 +00:00
tofunmi
8aa1638fe4 Add multi-image input FrameProcessor tests
PiperOrigin-RevId: 510441777
2023-02-24 09:14:39 +00:00
andrewlewis
bd4a5ecf6a Align test codec caps check with real implementation
PiperOrigin-RevId: 510437262
2023-02-24 09:11:12 +00:00
kimvde
2163cfb24a Move generateSilentAudio to Composition
Also rename to forceAudioTrack

PiperOrigin-RevId: 510394620
2023-02-17 14:22:50 +00:00
tofunmi
a4ad85d25a InternalTextureManager: delete texture after use
PiperOrigin-RevId: 510377977
2023-02-17 14:09:55 +00:00
huangdarwin
0cb5c58834 HDR: Catch test util decoder support error.
Otherwise, a lack of HDR decoding support will result in the tests checking output files for HDR output, like HdrEditingTest.transform_noRequestedTranscode_hdr10File_transformsOrThrows, failing.

PiperOrigin-RevId: 510213020
2023-02-17 13:57:07 +00:00
samrobinson
9ca1122bed Remove space in exception message.
This space was added by mistake in a prior CL of mine.

PiperOrigin-RevId: 510157363
2023-02-17 13:52:22 +00:00
huangdarwin
aa7abd5518 HDR: Add comment explaining force-SDR and AssetLoader interaction.
PiperOrigin-RevId: 510155318
2023-02-17 13:48:00 +00:00
huangdarwin
7d54f24473 HDR: Put force HDR as SDR into AssetLoader
This allows us to fix usage of HDR_MODE_EXPERIMENTAL_FORCE_INTERPRET_HDR_AS_SDR.

Before, this was checked in the VideoSamplePipeline, which no longer decides on the decoder configuration input format.

PiperOrigin-RevId: 510142097
2023-02-17 13:43:42 +00:00
kimvde
4cc5080db0 Rename occurrences of transform in lib-transformer
PiperOrigin-RevId: 510118760
2023-02-17 13:39:21 +00:00
andrewlewis
37768ed099 Remove Pixel watch check
It's unlikely anyone will try to use Transformer on watches.

PiperOrigin-RevId: 510115645
2023-02-17 13:34:56 +00:00
claincly
f74cd7b17e Fix a missed rename.
PiperOrigin-RevId: 510100709
2023-02-17 13:30:26 +00:00
andrewlewis
ab2cabe3da Check capabilities for more tests
Some of the test methods in `TransformationTest` don't check
capabilities. Add a check just scoped to decoding (not checking
encoding, but the default encoder factory may fall back).

PiperOrigin-RevId: 510088562
2023-02-17 13:25:57 +00:00
kimvde
3f84c86977 Rename TransformationTest to ExportTest
PiperOrigin-RevId: 510087543
2023-02-17 13:21:35 +00:00
huangdarwin
5fee7df622 Effects: Update javadoc to link to method.
PiperOrigin-RevId: 510077348
2023-02-17 13:17:10 +00:00
kimvde
774b44c6e2 Rename TransformationException to ExportException
PiperOrigin-RevId: 510062674
2023-02-17 13:12:44 +00:00
Googler
6867f56658 Update instrumentation to avoid making all constructors public
Previously, any constructors instrumented by Robolectric were made public. This
caused two types of issues:

1) If Android classes had non-public constructors which were made public and
added to the Android API, Robolectric allowed tests to incorrectly use the
constructors on older SDK levels (where they were non-public). This most
commonly occurs for AccessibiltyEvent and AccessibilityNodeInfo.

2) When reflection was used to instantiate classes that were instrumented by
Robolectric, all constructors were accessible, which did not match what
happened when running on an Android test.

Update the instrumentation in Robolectric to prevent making all public
constructors.

PiperOrigin-RevId: 510049123
2023-02-17 13:08:23 +00:00
huangdarwin
413f61b96d Effect: Rename FrameProcessor
Rename FrameProcessor to VideoFrameProcessor, and GlEffectsFrameProcessor to
DefaultVideoFrameProcessor.

Most changes are semi-mechanical, semi-manual find-replace, preserving case:
* "FrameProc" -> "VideoFrameProc" (ex. FrameProcessor -> VideoFrameProcessor, and
   FrameProcessingException -> VideoFrameProcessingException)
* "GlEffectsVideoFrameProc" -> "DefaultVideoFrameProc"

PiperOrigin-RevId: 509887384
2023-02-17 13:04:02 +00:00
andrewlewis
3e5ae92bc6 Fix some minor nits
PiperOrigin-RevId: 509879029
2023-02-17 12:59:36 +00:00
tofunmi
5168a35654 Change FrameProcessor.create() inputTrackType parameter to a boolean
PiperOrigin-RevId: 509808913
2023-02-15 18:28:54 +00:00
andrewlewis
6d9baa6e99 Skip tests if muxing is unsupported
PiperOrigin-RevId: 509802784
2023-02-15 18:24:47 +00:00
tofunmi
f7621aacc3 Move inputExternalSurfaceTexture to the ExternalTextureManager
Now that the GLEffectFrameProcessor handles external (video) and internal (image) input, components used only for external input needs should be moved to the ExternalTextureManager for code clarity

PiperOrigin-RevId: 509787494
2023-02-15 12:31:27 +00:00
samrobinson
7497b1e003 Use MediaFormatUtil for creating MediaFormat from Format.
PiperOrigin-RevId: 509785247
2023-02-15 12:27:43 +00:00
andrewlewis
3b7cf8f003 Add missing import for ExoPlayer v2 codebase
Issue: google/ExoPlayer#10989
PiperOrigin-RevId: 509771687
2023-02-15 10:49:11 +00:00
samrobinson
bcbe3f1f80 Clarify the mime type used in EncoderUtilTest.
PiperOrigin-RevId: 509750806
2023-02-15 10:45:30 +00:00
samrobinson
d91afa063a Align ASP and VSP on findSupportedMimeTypeForEncoder/Muxer logic.
* Moved the logic to SamplePipeline.
* Pass the requested values via Format.
* Moved exception throwing inside the methods.
* Build up the mimeTypesToCheck as a set - removing possible duplicate
checks.
* Simplified logic that calls the findSupportedMimeType method.
* Improved javadoc.

PiperOrigin-RevId: 509594062
2023-02-15 10:41:50 +00:00
samrobinson
ccd8856dac Preserve aspect ratio when scaling resolution is insufficient.
PiperOrigin-RevId: 509546771
2023-02-15 10:38:05 +00:00
huangdarwin
ccdb3b7028 Effect: Add leading zero to decimals in floats.
PiperOrigin-RevId: 509518734
2023-02-14 15:09:58 +00:00
christosts
b18dccde2a Version bump for ExoPlayer 2.18.3 & media3-1.0.0-rc01
#minor-release

PiperOrigin-RevId: 509501665
2023-02-14 13:49:22 +00:00
tofunmi
cdd924503a Fix InternalTextureManager so that all frames are sent downstream
PiperOrigin-RevId: 509478455
2023-02-14 11:23:29 +00:00
Googler
08cf6db305 Add exception cause to thrown exception
PiperOrigin-RevId: 509473556
2023-02-14 10:54:01 +00:00
claincly
efcb7683e5 Use ColorInfo.Builder in transformer and common.
Because the ColorInfo constructor is deprecated.

PiperOrigin-RevId: 509468663
2023-02-14 10:28:33 +00:00
claincly
33d45c3c75 Add error codes to reflect failures in using effects.
PiperOrigin-RevId: 509461955
2023-02-14 09:54:32 +00:00
huangdarwin
f9fffc28be Effect: Implement brightness convenience class.
It can be confusing how to use RgbMatrix for app devs not comfortable with image
manipulation, so add a helper class that simply translates rgb values the same
constant brightness value, to update the brightness values.

PiperOrigin-RevId: 509287229
2023-02-13 19:34:34 +00:00
samrobinson
dcd1f6bc6b Generalize getSupportedEncodersForHdr to return List<MediaCodecInfo>.
This now matches EncoderUtil.getSupportedEncoders return type.

PiperOrigin-RevId: 509222078
2023-02-13 15:33:02 +00:00
andrewlewis
77e651ad27 Update overlay test bitmap based on emulator output
PiperOrigin-RevId: 509210836
2023-02-13 14:35:38 +00:00
michaelkatz
711fa44d56 Catch IllegalArgumentExceptions in RTSP Response parsing
In parsing Describe RTSP response messages, IllegalArgumentExceptions are thrown for invalid parameters and values. These exceptions were not caught and crashed the Playback thread. Now these exceptions will be caught and their errors forwarded to the proper error handling listeners.

#minor-release

Issue: google/ExoPlayer#10971
PiperOrigin-RevId: 509207881
2023-02-13 14:18:53 +00:00
huangdarwin
e159741a95 Test: Move FileUtil to mh from mh.analysis.
FileUtil is only ever used in mh, and never in mh.analysis

PiperOrigin-RevId: 509206721
2023-02-13 14:12:46 +00:00
huangdarwin
15f0865d62 HDR: Remove HDR-specific error codes.
It may be confusing for 3P apps, for us to have separate error codes for (1) if
encoding HDR is not supported at all by the device, and (2) if encoding the
format, which happens to be HDR, is not supported by the device. Instead, we can
communicate this in the error message.

PiperOrigin-RevId: 509188666
2023-02-13 12:22:55 +00:00
andrewlewis
88a97fdb6f Fix audio generation in silence skipping test
Tests in `SilenceSkippingAudioProcessorTest` used half as many short integers as needed for channel values when generating alternating silence/noise input. Fix this by passing left and right channel input.

PiperOrigin-RevId: 509188074
2023-02-13 12:19:31 +00:00
claincly
117ce33af2 Add public API and render messages for setting effects.
PiperOrigin-RevId: 509165378
2023-02-13 10:15:28 +00:00