705 Commits

Author SHA1 Message Date
kimvde
f3ec59c87f Fix AS warnings and errors in FloatAudioMixingAlgorithmTest
When trying to run the test on Android Studio, error "incompatible
types: Buffer cannot be converted to ByteBuffer" is logged. This is
because ByteBuffer.flip() returns a Buffer (and not a ByteBuffer).
Annotation @CovariantReturnType on ByteBuffer.flip() should resolve this
automatically but it doesn't seem supported at the moment.
PiperOrigin-RevId: 496894723
2022-12-21 18:01:52 +00:00
kimvde
def6f64ea5 Simplify AssetLoader.Listener interface
Replace onTrackRegistered and onAllTracksRegistered with onTrackCount.

PiperOrigin-RevId: 496853037
2022-12-21 17:50:04 +00:00
huangdarwin
ef35170e21 HDR: Add Transformer GL tone mapping test.
PiperOrigin-RevId: 496727803
2022-12-21 17:42:17 +00:00
huangdarwin
dea7ab314a HDR: Implement Transformer HDR to SDR GL tone-mapping API
Note that we simply use GlEffectsFrameProcessor in-app / GL tone-mapping, so PQ->SDR tone-mapping isn't yet implemented.

Tested manually using the demo on Pixel 7, to confirm that device and in-app tone
mapping behave similarly.

PiperOrigin-RevId: 496700231
2022-12-21 17:34:35 +00:00
steveanton
3481d4a15e Add low level audio mixing algorithms
Adds the AudioMixerAlgorithm interface which allows for specialized
implementations of audio mixing that also efficiently convert between
source and mixing formats.

Initial implementation has two algorithms:
1. Float -> float (with channel mixing)
2. S16 -> float (with channel mixing)

PiperOrigin-RevId: 496686805
2022-12-21 17:30:45 +00:00
tofunmi
b055010c11 Create local fallbackDetails to remove checkNotNull.
PiperOrigin-RevId: 496664711
2022-12-21 17:22:55 +00:00
tofunmi
987354e5b5 Add fallback details to analysis json.
PiperOrigin-RevId: 496660388
2022-12-21 17:15:08 +00:00
kimvde
18031e25bb Make AssetLoader progress not Transformer specific
To do that, rename PROGRESS_STATE_NO_TRANSFORMATION to
PROGRESS_STATE_NOT_STARTED and update Javadoc of ProgressState to not be
Transformer specific.

PiperOrigin-RevId: 496653460
2022-12-21 17:11:15 +00:00
andrewlewis
14517e463d Fix local asset naming
PiperOrigin-RevId: 496410502
2022-12-21 17:07:21 +00:00
huangdarwin
597fa3f694 HDR: Use local instead of remote file for testing.
This should allow us to focus on HDR failures instead of network buffering
failures when debugging HDR issues.

These files are each used on several files, so it should be more worth the
test binary impact to move these files to local first.

Locally, tests did take less time after this diff

PiperOrigin-RevId: 496398130
2022-12-21 17:03:28 +00:00
kimvde
cd70a5cef5 Move setMediaSourceFactory to ExoPlayerAssetLoader
The MediaSourceFactory won't be used by the other AssetLoaders

In order to do that, ExoPlayerAssetLoader has been made public, and the
DefaultAssetLoaderFactory has become a wrapper around
ExoPlayerAssetLoader.

PiperOrigin-RevId: 496386853
2022-12-21 16:59:35 +00:00
tonihei
a032da58a4 Rollback of c566b77449
*** Original commit ***

Add TransformerTestBuilderFactory to make transformer testable by apps

***

PiperOrigin-RevId: 496342997
2022-12-21 16:36:23 +00:00
samrobinson
e4695048a8 Make clear video encoder HighQualityTargeting is experimental.
PiperOrigin-RevId: 495860300
2022-12-21 16:20:47 +00:00
samrobinson
aecc810937 Pass required value into TransformationTestResult.Builder constructor
PiperOrigin-RevId: 495842393
2022-12-21 16:05:22 +00:00
tofunmi
c566b77449 Add TransformerTestBuilderFactory to make transformer testable by apps
PiperOrigin-RevId: 495821660
2022-12-21 16:01:31 +00:00
samrobinson
e46b4c0fdc Create and return a TransformationResult regardless of success.
The TransformationResult has some useful values that are set in error
cases, such as the codecs used.

PiperOrigin-RevId: 495568259
2022-12-15 17:15:59 +00:00
claincly
678b29c10d Make Codec release frames to frame processor.
PiperOrigin-RevId: 495406734
2022-12-15 17:03:52 +00:00
kimvde
254842b782 Pass CapturingDecoderFactory to AssetLoader
Otherwise, the decoders are not captured. It works at the moment for the
video decoder because decoding is still done on the sample pipeline but
it will moved to the AssetLoader soon.

PiperOrigin-RevId: 495275575
2022-12-15 16:44:04 +00:00
andrewlewis
58242e6caf Output direct buffers from SilentAudioGenerator
`AudioProcessor`s expect direct buffers. This shouldn't make any functional difference in our code, but a custom audio processor might try to access the buffer from JNI in which case a direct byte buffer is more efficient.

PiperOrigin-RevId: 495241669
2022-12-15 16:40:09 +00:00
andrewlewis
eb95362029 Take format for SilentAudioGenerator
This simplifies the caller slightly.

PiperOrigin-RevId: 495234339
2022-12-15 16:36:15 +00:00
claincly
42f0850fa5 Fix method name to return all encode-able MIME types
PiperOrigin-RevId: 495093939
2022-12-15 16:32:18 +00:00
kimvde
339ce4fced Add AssetLoader.Factory
This is so that apps can customise AssetLoader

PiperOrigin-RevId: 494998497
2022-12-15 16:17:00 +00:00
andrewlewis
c17c23d1f1 Generate complete silent audio frames
`SilentAudioGenerator` could output a fractional audio frame, and this could cause downstream components to throw because of trying to read a complete audio frame but only seeing a partial one.

Calculate the output buffer size based on the frame size (which is a no-op for stereo 16-bit audio) and calculate a total number of frames to output then multiple by the frame size.

PiperOrigin-RevId: 494992941
2022-12-15 16:13:05 +00:00
samrobinson
03d6da8528 Add codec names to TransformationResult
PiperOrigin-RevId: 494736085
2022-12-15 16:01:01 +00:00
samrobinson
b18d34fc58 Move SetFrameEditTest into TransformationTest.
PiperOrigin-RevId: 494651985
2022-12-12 14:57:31 +00:00
kimvde
0c8c7eeebd Add AssetLoader interface
This is a step towards allowing apps to inject a custom AssetLoader

PiperOrigin-RevId: 494185078
2022-12-12 14:53:17 +00:00
kimvde
1d0425e211 Make input media duration volatile
The duration in TransformerInternal.ComponentListener is set on the
Transformer internal thread, and is read on the playback thread. Making
this field volatile ensures that the playback thread reads the updated
value.

PiperOrigin-RevId: 493908385
2022-12-12 14:36:24 +00:00
huangdarwin
e42a438a3d Transformer: Always initialize presentationTime in VTSP.
This shouldn't really have anything to do with HDR.

PiperOrigin-RevId: 493357915
2022-12-12 13:45:31 +00:00
huangdarwin
e9a3077371 Transformer: Rename error code from output to encoding format.
Rename ERROR_CODE_OUTPUT_FORMAT_UNSUPPORTED to
ERROR_CODE_ENCODING_FORMAT_UNSUPPORTED.

This makes the error code more consistent with ERROR_CODE_DECODING_FAILED on the
decoding side. Also, the error code is in the "Encoding errors (4xxx)" section,
so muxer errors probably should be in the "Muxer errors (7xxx)" section instead.

Additionally, no muxer errors currently seem to use
ERROR_CODE_OUTPUT_FORMAT_UNSUPPORTED, so this should be a safe change.

PiperOrigin-RevId: 493322880
2022-12-12 13:36:56 +00:00
kimvde
5277171d0b Move slow mo video flattening to AssetLoader
PiperOrigin-RevId: 493300556
2022-12-12 13:32:46 +00:00
kimvde
8d655e1267 Move progress updates to the AssetLoader
This is necessary to move video slow motion flattening to the
AssetLoader because this step can change the duration. As we use the
duration before flattening to calculate the progress, we must also use
the position before flattening.

PiperOrigin-RevId: 493291990
2022-12-12 13:28:30 +00:00
claincly
561998ab6a Re-assign TODOs
PiperOrigin-RevId: 493281221
2022-12-12 13:11:33 +00:00
ibaker
c006575d43 Add javadoc links to README files
Fix some other link titles and destinations spotted along the way.

#minor-release

PiperOrigin-RevId: 493276172
2022-12-12 13:07:19 +00:00
kimvde
f14a7f0999 Add comment to explain why cancel exceptions are handled differently
PiperOrigin-RevId: 493260798
2022-12-12 12:58:51 +00:00
huangdarwin
cf61e219ef HDR: Add effect interface support for different in/out color transfers
Modify FrameProcessor and MatrixTextureProcessor interfaces to support
different input and output color transfers. Does not implement conversion between
color ranges (ex. HDR and SDR), but should allow for conversion between color
transfers of the same color range (ex. HLG and PQ).

This supports in-app tone mapping, where we need a single FrameProcessor to
input HDR color transfers (ex. HLG/PQ) and output SDR (ex. gamma2.2). This also
supports previewing, where we need a single FrameProcessor to be able to input HLG
and output PQ.

Manually tested by confirming colors still look right on SDR and HDR videos
with a rotation and color affect applied.

PiperOrigin-RevId: 493108678
2022-12-12 12:54:38 +00:00
claincly
88a7d67f4a Change argument order to match API council suggestion
By putting Listener as the last argument.

PiperOrigin-RevId: 493100906
2022-12-12 12:50:21 +00:00
kimvde
72773a862e Avoid sending message on dead thread
PiperOrigin-RevId: 493040714
2022-12-12 12:46:05 +00:00
claincly
656b346e6c Allow specifying an Executor in FrameProcessor.Factory.create()
From this CL on, FrameProcessor listeners will be invoked from an Executor that
is passed in when creating the FrameProcessor.

GlTextureProcessor needs to invoke the ErrorListener on the said Executor too.

PiperOrigin-RevId: 493018583
2022-12-12 12:41:47 +00:00
tonihei
f3fc4fb973 Fix threading of onFallbackApplied callback
The callback is currently triggered on the ExoPlayer playback thread
instead of the app thread that added the listener.

PiperOrigin-RevId: 492474405
2022-12-12 12:20:40 +00:00
kimvde
c42fc6f33f Make sure that the sample pipeline data is processed regularly.
This is necessary to move video decoding to the AssetLoader. Otherwise,
if the decoder max pending frame count is reached, the AssetLoader will
stop queuing frames to the pipeline, and process data will not be called
anymore.

PiperOrigin-RevId: 492392621
2022-12-12 12:03:43 +00:00
samrobinson
16ea364478 Merge feedXFromInput and feedXFromSilence in ATSP.
PiperOrigin-RevId: 492195931
2022-12-12 11:58:57 +00:00
samrobinson
5d2cc8dc36 Add an E2E test for changing sample rate with AudioProcessor.
PiperOrigin-RevId: 492160193
2022-12-12 11:41:47 +00:00
samrobinson
32bed82897 Add an option to force a silent audio track.
PiperOrigin-RevId: 492154544
2022-12-12 11:37:35 +00:00
claincly
e4db710deb Fix Samsung MediaCodec encoder reports incorrect timestmp in EOS
PiperOrigin-RevId: 492023573
2022-12-12 11:15:41 +00:00
huangdarwin
2652d0efd7 Transformer: Move getMaxPendingFrameCount logic to constructor.
Also, use the mediaCodecName passed into the constructor to get the
maxPendingFrameCount.

PiperOrigin-RevId: 491985044
2022-12-12 11:07:04 +00:00
claincly
c308802ad1 Add function to clear the cached list of encoders
EncoderUtil using a static list to cache encoders means it's not possible to
use a different set of encoders for different tests when running all
robolectric tests together.

PiperOrigin-RevId: 491959350
2022-12-12 10:58:04 +00:00
kimvde
ed20420b9b Move audio decoding to AssetLoader
PiperOrigin-RevId: 491933937
2022-12-12 10:53:48 +00:00
huangdarwin
4496cf551f HDR: Limit Z Fold 4 c2.qti decoder pending frame count.
Bypass Z Fold 4 HDR10 tone-mapping bug by limiting the max frame count to 12.

This passed with a value of 14, and failed with a value of 15, but I figured I'd use 12 just to be safe.

PiperOrigin-RevId: 491684058
2022-12-12 10:41:20 +00:00
samrobinson
349144f65d Change TransformerInternal durationMs to durationUs.
Follow-up CLs will use this duration for silent audio.

PiperOrigin-RevId: 491670359
2022-11-29 19:57:09 +00:00
claincly
18280723be Move audio MIME type fallback away to ATSP
PiperOrigin-RevId: 491660842
2022-11-29 19:53:30 +00:00