2138 Commits

Author SHA1 Message Date
samrobinson
320d9a7ed2 Verify the audio properties being used within Transformer.
Add TransformerAudioEndToEndTest to emulated tests.

PiperOrigin-RevId: 514991434
2023-03-14 07:37:26 +00:00
tianyifeng
e052b63514 Handle dead object error during writing buffer to AudioTrack
PiperOrigin-RevId: 514989776
2023-03-14 07:36:43 +00:00
kimvde
67d15ec575 Fix threading issueis in SequenceAssetLoaderListener
Callbacks onTrackCount and onTrackAdded can be called simultaneously
from different threads.

Before this fix, it was possible for the MuxerWrapper and
FallbackListener track count to never be set, or to be set
with incorrect values.

PiperOrigin-RevId: 514779719
2023-03-14 07:35:25 +00:00
tonihei
62c0352416 Use current position code when checking if AudioTrack has pending data
We currently rely on the raw playback head position to check if any
data is pending in the AudioTrack (e.g. to know if the renderer is
still ready). We can use the value returned from getCurrentPositionUs
instead to align the "isReady" logic with the playback position logic.

This has the side effect that getPlaybackHeadPosition position is called
less often when the position is obtained via getTimestamp.

PiperOrigin-RevId: 514747613
2023-03-14 07:34:14 +00:00
tofunmi
e478d81b52 Update effect to take in and use a GlObjectsProvider
PiperOrigin-RevId: 514744747
2023-03-14 07:33:25 +00:00
tonihei
20669fca27 Document that custom commands can only be sent if they are defined
Custom commands from controller to session are blocked if they are
not listed in the available session command list. This isn't well
documented in the Javadoc currently.

#minor-release

PiperOrigin-RevId: 514733568
2023-03-14 07:32:36 +00:00
tonihei
788132b9dc Make DefaultDashChunkSource more robust against empty segment timelines
Segment timelines are technically allowed to be empty, but not all places
add the necessary checks.

Issue: google/ExoPlayer#11014
PiperOrigin-RevId: 514722205
2023-03-07 15:52:07 +00:00
samrobinson
f07e6b63ea Fix AudioMixer javadoc.
PiperOrigin-RevId: 514699994
2023-03-07 15:50:33 +00:00
huangdarwin
fe7f3d51df GL: Reduce visibility of GlProgram.loadAsset.
This is only used within the same class, so can be private.

PiperOrigin-RevId: 514665000
2023-03-07 11:57:41 +00:00
samrobinson
3264cb8271 Pass Metadata to Muxer when adding a track.
PiperOrigin-RevId: 514575400
2023-03-07 11:56:53 +00:00
tofunmi
7303caffb5 Create GlObjectsProvider
To create this file TextureInfo has been moved to common and renamed to GLTextureInfo.

We'll look to expand the interface in future to cover more of the methods around GL object maintenance in future as required.

PiperOrigin-RevId: 514445397
2023-03-07 11:56:08 +00:00
kimvde
5115df1147 Rename CompositeAssetLoader to SequenceAssetLoader
This clarifies that a SequenceAssetLoader loads data corresponding to an
EditedMediaItemSequence.

PiperOrigin-RevId: 514442681
2023-03-07 11:55:22 +00:00
tofunmi
fe38901028 ImageAssetLoaderTest minor fixes
PiperOrigin-RevId: 514436359
2023-03-07 11:54:36 +00:00
tofunmi
f4b88cd8a2 Add ImageAssetLoader tests.
PiperOrigin-RevId: 514431184
2023-03-07 11:53:36 +00:00
kimvde
0f8b67b875 Make onOutputFormat nullable
- This is to make sure we know about all the tracks before initializing
the SamplePipelines. This allows to set the muxer and the fallback
listener track count before the SamplePipelines are built.
- As a result, the test files had to be updated because the order in
which the tracks are written has changed.
- The ImageAssetLoader also had to be updated to call onOutputFormat
repeatedly until it returns a non-null SampleConsumer.
- Also fix the trackCount sent to the muxer and fallback listener. The
correct track count can be computed now that we know about all the
tracks before building the SamplePipelines.

PiperOrigin-RevId: 514426123
2023-03-07 11:52:52 +00:00
huangdarwin
729066fbd6 GL: Delete frame buffers after use.
Before, we used to never call glDeleteFramebuffers, which could
in theory lead to leaks in the number of frame buffers
available and make releasing the GL context more expensive.

PiperOrigin-RevId: 514387847
2023-03-07 11:52:01 +00:00
sheenachhabra
a81a5f2699 Create orientation matrix byte array using util method
PiperOrigin-RevId: 514375835
2023-03-07 11:51:13 +00:00
ibaker
cb7d565fd4 Add missing @param tags
Dackka generates a warning if a method has at least one `@param` tag,
but not all of them are documented.

#minor-release

PiperOrigin-RevId: 513873453
2023-03-07 11:49:24 +00:00
ibaker
c0459e6ae1 Exclude test_session_{common,current} from API and Javadoc tests
We don't publish documentation for these, nor maven artifacts for them -
they only exist as source code on GitHub.

PiperOrigin-RevId: 513863141
2023-03-07 11:48:38 +00:00
ibaker
5d23a92923 Ensure ForwardingPlayer users do listener registration correctly
The `@CallSuper` annotation should help catch cases where subclasses are
calling `delegate.addListener` instead of `super.addListener` but it
will also (unintentionally) prevent subclasses from either completely
no-opping the listener registration, or implementing it themselves in a
very custom way. I think that's probably OK, since these cases are
probably unusual, and they should be able to suppress the warning/error.

Issue: androidx/media#258

#minor-release

PiperOrigin-RevId: 513848402
2023-03-07 11:47:49 +00:00
tofunmi
efaf4e3f33 Remove effect/SimpleBitmapLoader & replace with DataSourceBitmapLoader
PiperOrigin-RevId: 513824487
2023-03-07 11:46:58 +00:00
kimvde
71fb4f9a5c Make sure that tracks are transcoded/transmuxed consistently
All audio tracks should either all be transcoded or all be transmuxed.
Same for video tracks.

To achieve this, simplify the behaviour of transmuxAudio/Video.

PiperOrigin-RevId: 513809287
2023-03-07 11:46:07 +00:00
samrobinson
07ba24ab1a Provide TransformerInternal with the AssetLoader output format.
TI can use this Format as part of creating the SamplePipelines.

PiperOrigin-RevId: 513777622
2023-03-07 11:45:16 +00:00
sheenachhabra
f23d6c1dc1 Fix nit issues in muxer end to end tests
Renamed MuxerEndToEndTest.java to Mp4MuxerEndToEndTest.java to align it with class under test.
Removed muxed prefix from dump file name because Mp4 implicitely means muxed only.

PiperOrigin-RevId: 513574681
2023-03-07 11:44:26 +00:00
tonihei
8498e4b444 Fix lint-baseline.xml for latest UI translations
#minor-release

PiperOrigin-RevId: 513533248
2023-03-02 17:21:16 +00:00
andrewlewis
82119bbbc2 Fix typo
PiperOrigin-RevId: 513529059
2023-03-02 17:20:30 +00:00
andrewlewis
a28b691ced Fix name for FrameCacheGlShaderProgram
This implements `GlShaderProgram` (and is GL-specific).

PiperOrigin-RevId: 513528160
2023-03-02 17:19:29 +00:00
samrobinson
ae4471debf Add getNormalizedTrackType, mapping image to video track type.
PiperOrigin-RevId: 513514142
2023-03-02 17:18:39 +00:00
bachinger
6042bec18a Fix some JavaDoc in the Media3 session module
#minor-release

PiperOrigin-RevId: 513501046
2023-03-02 13:26:01 +00:00
tonihei
cd753bd7b8 Bump version numbers to Media3 1.0.0-rc02 and ExoPlayer 2.18.4
#minor-release

PiperOrigin-RevId: 513488487
2023-03-02 13:25:15 +00:00
tofunmi
8ec1f86967 README.md line reflow.
PiperOrigin-RevId: 513289716
2023-03-02 13:23:30 +00:00
tofunmi
e425b6e082 Image frame processing color transfer fix
Used an actual captured image with set color profile for test to minimise the chance of the test flaking. Also renamed the media/bitmap/overlay folder to media/bitmap/input_images for clarity.

PiperOrigin-RevId: 513273353
2023-03-02 13:22:27 +00:00
samrobinson
65afd40622 Only init SpeedChangingAudioProcessor if Format.Metadata not null.
If the Metadata passed to SegmentSpeedProvider is null, then the
SegmentSpeedProvider will always return 1f from getSpeed.

Initializing a SpeedChangingAudioProcessor requires a SpeedProvider.
Once configured,this audioProcessor is always active, so buffers are
passed through it. Because getSpeed is always 1, the processor performs
a no-op, but still has to do a buffer copy for each buffer.

By not initializing the audio processor when metadata is null, this
copy can be skipped and the audio pipeline is more performant.

Note: This change does not affect the multiple media-item case, which
is not supported with speed changes, as per Transformer API
documentation.
PiperOrigin-RevId: 513261811
2023-03-02 13:21:41 +00:00
sheenachhabra
9f51182bb8 Move Mp4MuxerMetadataTest into muxer module
Changes include:
1. Move the test fine into muxer module.
2. Use dump file infra for test cases.
3. Add one additional test for adding float metadata.
4. Few improvements in the code.

In next CL will remove Mp4 term from the file name as we are not using this term in test file names.

PiperOrigin-RevId: 513222506
2023-03-01 17:27:58 +00:00
tonihei
17499cefcc Merge pull request #255 from mayurk2:use_edts_offset_if_it_is_for_entire_file
PiperOrigin-RevId: 513213229
2023-03-01 17:27:15 +00:00
ibaker
1c4ba5d9f6 Delete VorbisUtil.CodeBook since it's not used
PiperOrigin-RevId: 513186205
2023-03-01 17:26:31 +00:00
ibaker
c07cf3dc41 Remove @see tags with <a> tags
These are not supported by Dackka

#minor-release

PiperOrigin-RevId: 513176533
2023-03-01 17:25:48 +00:00
andrewlewis
cccc582595 Don't log removed encoders
We shouldn't have this logging unless we really need it to debug
a specific problem, as it can be noisy (even at debug level).

PiperOrigin-RevId: 512904412
2023-03-01 17:25:03 +00:00
christosts
42fae152d0 Minor change in ForwardingPlayer javadoc
#minor-release

PiperOrigin-RevId: 512897269
2023-03-01 17:23:40 +00:00
tonihei
a7faa5bfd8 Update translations
#minor-release

PiperOrigin-RevId: 512890813
2023-03-01 17:22:55 +00:00
kimvde
d2d5174f09 Handle multiple sequences in a Composition
This lays the groundwork for full multi-asset, and more particularly for
adding looping background audio.

PiperOrigin-RevId: 512887888
2023-03-01 17:22:10 +00:00
tonihei
408b4449ff Ensure getPlaybackHeadPosition isn't called if not needed
Once the value returned from AudioTimestampPoller advances, we
only need getPlaybackHeadPosition to sample sync params and
verify the returned timestamp. Both of these happen less often
and we can avoid calling getPlaybackHeadPosition if we don't
actually need it.

PiperOrigin-RevId: 512882170
2023-03-01 17:21:27 +00:00
andrewlewis
5b3f3e649a Increase GL release timeout to 500 ms
Based on 1000 test runs an emulator, with the current timeout releasing
fails (even with no custom effects) about one percent of the time.
Releasing normally completes in about 30 ms but occasionally
`eglTerminate` took up to 200 ms (and even releasing an effect
took up to 80 ms in one case).

With the new timeout of 500 ms, we still catch stuck effects reasonably
quickly but the number of flaky test failures should be less than one in
ten thousand.

PiperOrigin-RevId: 512690715
2023-03-01 17:20:37 +00:00
bachinger
1760d63fc4 Add FakeMultiPeriodLiveTimeline and test case
This timeline will be used in unit test cases of follow-up
CLs. It basically can be used to emulate the timeline created by a
multi-period live media source when the real time advances.

PiperOrigin-RevId: 512665552
2023-03-01 17:19:57 +00:00
tofunmi
198d73425e Create DataSourceBitmapLoader
PiperOrigin-RevId: 512659747
2023-03-01 17:19:13 +00:00
tonihei
e79b47ccff Fix some playback parameter signalling problems.
Playback parameter signalling can be quite complex because
 (a) the renderer clock often has a delay before it realizes
     that it doesn't support a previously set speed and
 (b) the speed set on media clock sometimes intentionally
     differs from the one surfaced to the user, e.g. during
     live speed adjustment or when overriding ad playback
     speed to 1.0f.

This change fixes two problems related to this signalling:
 1. When resetting the media clock speed at a period transition,
    we don't currently tell the renderers that this happened.
 2. When a delayed speed change update from the media clock is
    pending and the renderer for this media clock is disabled
    before the change can be handled, the pending update becomes
    stale but it still applied later and overrides any other valid
    speed set in the meantime.

Both edge cases are also covered by extended or new player tests.

Issue: google/ExoPlayer#10882

#minor-release

PiperOrigin-RevId: 512658918
2023-03-01 17:18:30 +00:00
sheenachhabra
36aa29809d Add Muxer end to end tests
Following test cases are added:
1. Mux H264 video
2. Mux H265 video
3. Mux HDR video

Each test case performs following actions:
1. Extract track and samples from input Mp4 using MediaExtractor.
2. Feed those samples into Mp4 muxer.
3. Use extractor to extract the samples from muxed file and create dump file.

PiperOrigin-RevId: 512589069
2023-02-27 18:50:56 +00:00
tonihei
3970343846 Ensure output format is updated in sync with stream changes.
MediaCodecRenderer currently has two independent paths to trigger
events at stream changes:
 1. Detection of the last output buffer of the old stream to trigger
    onProcessedStreamChange and setting the new output stream offset.
 2. Detection of the first input buffer of the new stream to trigger
    onOutputFormatChanged.
Both events are identical for most media. However, there are two
problematic cases:
  A. (1) happens after (2). This may happen if the declared media
     duration is shorter than the actual last sample timestamp.
  B. (2) is too late and there are output samples between (1) and (2).
     This can happen if the new media outputs samples with a timestamp
     less than the first input timestamp.

This can be made more robust by:
 - Keeping a separate formatQueue for each stream to avoid case A.
 - Force outputting the first format after a stream change to
   avoid case B.

Issue: google/ExoPlayer#8594

#minor-release

PiperOrigin-RevId: 512586838
2023-02-27 18:50:03 +00:00
tonihei
d0cbf0fce8 Add workaround for wrong PerformancePoints on some devices.
Some devices were reported to have wrong PerformancePoint sets
that cause 60 fps to be marked as unsupported even though they
are supported.

Issue: google/ExoPlayer#10898

#minor-release

PiperOrigin-RevId: 512580395
2023-02-27 18:49:00 +00:00
tonihei
7ffcc6f7ea Correctly update output info if previous stream has been fully rendered
The output info for a new stream is marked pending until the last
sample of the previous stream has been processed. However, this fails
if the previous stream has already been fully processed. We need to
detect this case explicitly to avoid signalling the output change one
sample too late.

#minor-release

PiperOrigin-RevId: 512572854
2023-02-27 18:47:20 +00:00