1208 Commits

Author SHA1 Message Date
sheenachhabra
dc865e6e98 Return TIME_UNSET for last sync sample if there is no video track
PiperOrigin-RevId: 565395263
2023-09-14 09:44:42 -07:00
claincly
bf25b3e89d Move listener methods into private method for readability
The logic that handles components' boundaries are grouped together in private
methods, like handling VideoCompositor's output textures.

PiperOrigin-RevId: 565131579
2023-09-13 12:45:40 -07:00
claincly
1d8135e563 Add test for MultiInputVideoGraph
This test composites the first frame from two video inputs.

PiperOrigin-RevId: 565090338
2023-09-13 10:28:07 -07:00
claincly
5106f2f45a Rollback of e2882c051b
PiperOrigin-RevId: 565069442
2023-09-13 09:15:35 -07:00
Googler
282171cb6f Rollback of e2882c051b
PiperOrigin-RevId: 564802194
2023-09-12 12:40:57 -07:00
claincly
e2882c051b Implement MultipleInputVideoGraph
Also adds DebugTraceUtil category to track MultipleInputVideoGraph events.

PiperOrigin-RevId: 564749202
2023-09-12 09:45:23 -07:00
sheenachhabra
b042943102 Add different modes in MuxerWrapper
For pause and resume feature we need to use same `MuxerWrapper`
for `remuxing processed video` and then to `process remaining video`.
In order to use same `MuxerWrapper` across `different Exports`
we need to preserve its state.

PiperOrigin-RevId: 564728396
2023-09-12 08:28:35 -07:00
christosts
4afecc9099 Rollback of 51fd06482b
PiperOrigin-RevId: 563714392
2023-09-08 04:39:51 -07:00
samrobinson
866d62dd34 Improve CompositionExportTest assertions by using dump files.
PiperOrigin-RevId: 563708666
2023-09-08 04:09:21 -07:00
samrobinson
763dddfbd4 Dump with C.TrackType as key, rather than muxer track index.
Modifying dumping to not required "released" to be called.

Track index is an arbitrary value based on the order of addTrack calls.
Samples are dumped by track (rather than as soon as they are written),
so it's preferable to use a value that provides more context.

By using the track type as a key, dump files will be more deterministic
and will have more similarities when branched.

PiperOrigin-RevId: 563700982
2023-09-08 03:25:13 -07:00
huangdarwin
4c792ac907 Compositor: Add API for specifying input texture ColorInfo.
That said, only SDR is supported for now, so this will always throw if
HDR is input. This will also throw if different ColorInfo values are input
because color SDR mixing (ex. between sRGB and BT709) is not yet supported.

PiperOrigin-RevId: 563457729
2023-09-07 09:38:41 -07:00
sheenachhabra
3835aa5344 Build ExportResult in Transformer.java
For pause and resume feature we will need to build `ExportResult`
from multiple internal Exports. Keeping the `ExportResultBuider` in
`Transformer` class will allow using same builder across different
internal exports.

PiperOrigin-RevId: 563392443
2023-09-07 04:48:52 -07:00
samrobinson
cff2816da4 Add silence generation parameterized test case.
PiperOrigin-RevId: 563098931
2023-09-06 07:34:32 -07:00
samrobinson
92814b84a8 Add alternate MP4 asset with PCM audio.
Audio is Mono 44.1kHz.

Created using:
`ffmpeg -i <input> -c:v copy -c:a pcm_s16be <output>`

PiperOrigin-RevId: 563079553
2023-09-06 05:58:27 -07:00
samrobinson
ef45c0fe5d Add parameterized dump tests for single item exports.
PiperOrigin-RevId: 563075771
2023-09-06 05:35:51 -07:00
andrewlewis
b56bdd8da0 Remove unnecessary track group count assertion
When checking the color transfer, there is no reason that the file should have
exactly two tracks, and this assertion means that this method can't be used
as-is for checking video-only files, for example.

PiperOrigin-RevId: 562813111
2023-09-05 10:06:44 -07:00
sheenachhabra
657856dfe5 Fix/Update TODOs in Mp4Muxer tests
PiperOrigin-RevId: 562728777
2023-09-05 04:04:51 -07:00
huangdarwin
2bdda35731 Overlay: Rename videoFrameAnchor to backgroundFrameAnchor.
Overlays may be overlaid over:
* In VideoFrameProcessor, videos or images (or texture input).
* In Compositor, other videos.

In Compositor, Overlays may consist of video, so it could be confusing
for videoFrameAnchor to contrast with overlayAnchor.

Also, rename overlayAnchor to overlayFrameAnchor, since it's modifying
the anchor in the overlay's frame, so this name seems slightly more precise.

PiperOrigin-RevId: 562004292
2023-09-01 11:28:19 -07:00
tofunmi
502504fd47 Skip runUntilComparisonFrameOrEnded checks for nexus's on API 21
verified with nexus 5 running on api 21

PiperOrigin-RevId: 561957198
2023-09-01 08:10:02 -07:00
huangdarwin
57bc215210 Compositor: Implement OverlaySettings and custom in/out size support.
Implement VideoCompositor support of:
* Different input and output sizes
* CompositorSettings, to customize output size based on input texture sizes
* OverlaySettings, to place an input frame in an arbitrary position on
  the output frame.

Also, refactor Overlay's matrix logic to make it more reusable between
Compositor and Overlays

PiperOrigin-RevId: 561931854
2023-09-01 05:54:15 -07:00
claincly
6c2713f153 Renames containsKey to contains so it's more readable
PiperOrigin-RevId: 561916837
2023-09-01 04:25:25 -07:00
sheenachhabra
219b253731 Pass initial timestamp offset to VideoFrameProcessor
For pause and resume feature we will remux the previously processed
video in the first export and then in the second export we will resume
processing remaining video. For the second export we will have to
set initial timestamp offset so that video samples from the second
export are in continuation to the previously muxed samples.

PiperOrigin-RevId: 561689651
2023-08-31 10:17:40 -07:00
sheenachhabra
56acb3ee21 Move start export code into startInternal() method
For pause and resume we need to perform multiple intermediate exports.
So moved start() logic into a separate so that it can be reused from
different places with different Compositions and MuxerWrapper.

PiperOrigin-RevId: 561675472
2023-08-31 09:26:02 -07:00
samrobinson
6ecd3e9c2d Dump DumpableSamples list in track order.
Switch to SparseArray<List> rather than a Map<Integer, List>.

Track indices now determine sample list dumping, reducing flakiness.

PiperOrigin-RevId: 561658586
2023-08-31 08:15:41 -07:00
sheenachhabra
b37e37aa3c Pass MuxerWrapper to the TransformerInternal
For pause/resume feature, same MuxerWrapper needs to be
used across intermediate exports. So pass the MuxerWrapper
from Transformer.java
More specifically, when resume() is called
1. Create a MuxerWrapper and remux the previous
video samples (Export 1).
2. User the same MuxerWrapper and start processing
remaining video samples (Export 2).

PiperOrigin-RevId: 561325867
2023-08-30 06:45:39 -07:00
claincly
b466b06ace Split out VideoGraph and VideoFrameProcessingWrapper
More specifically, this CL

- Defines a VideoGraph interface
  - Adds a factory method, to hide the constructors
  - Separate out an initialize method that does the real work (following that constructor should not do real work)
  - The VideoGraph takes in a list of composition effects. For now, we only use the `Presentation`, if there is one in the list. This means we can take any number of Presentations now, but only the first one will be used.
- Moves the VideoFrameProcessingWrapper to its own file

PiperOrigin-RevId: 561059653
2023-08-29 09:54:43 -07:00
tofunmi
667103f2bd Fix SequenceAssetLoader signalling End Of Video Input twice
PiperOrigin-RevId: 560170216
2023-08-25 12:46:50 -07:00
sheenachhabra
1f013afea8 Add Mp4ExtractorWrapper into Transformer module
For pause and resume feature we need to find the timestamp of
the last sync sample in an MP4 file. The Mp4ExtractorWrapper
provides an easy to use API on top of Mp4Extractor.

PiperOrigin-RevId: 560113112
2023-08-25 09:29:28 -07:00
huangdarwin
01689ba2ec Compositor: Queue bitmaps by timestamp.
This allows us to manage timestamps in one list, instead of both via
offsets/framerates, and a list.

PiperOrigin-RevId: 560053888
2023-08-25 04:44:48 -07:00
claincly
feae0245b9 Remove the textureReleaseCallback from textureOutputListerner
PiperOrigin-RevId: 559817280
2023-08-24 11:33:11 -07:00
claincly
28fd43617e Factor out VideoFrameProcessor logic
PiperOrigin-RevId: 559780905
2023-08-24 09:42:21 -07:00
huangdarwin
cd0b7d9e29 Transformer: Document not supporting setTextureOutput on VFP Factory.
PiperOrigin-RevId: 559760003
2023-08-24 08:25:33 -07:00
claincly
8c1bedb483 Remove unnecessary types in lambda
PiperOrigin-RevId: 559745329
2023-08-24 07:21:52 -07:00
sheenachhabra
a6f9fefb88 Add setSequences method to Composition.Builder
For pause and resume feature, we need to buildUpon the
original composition and then set the modified sequences.

PiperOrigin-RevId: 559723694
2023-08-24 05:35:06 -07:00
claincly
930e538aca Allow re-configuring DVFP in a (mostly) non-blocking manner
The first call to method `registerInputStream` doesn't block.

Later successive calls to the method blocks until the previous register call
finishes.

PiperOrigin-RevId: 559694490
2023-08-24 02:55:54 -07:00
sheenachhabra
5276f797e5 Add buildUpon() and setMediaItem() method in EditedMediaItem
For pause/resume feature we need to build upon original EditedMediaItem
and set a modified MediaItem to it.

PiperOrigin-RevId: 559484129
2023-08-24 09:32:09 +01:00
tofunmi
404a259295 Remove queueInputBitmap by framerate and duration interfaces
PiperOrigin-RevId: 559384011
2023-08-24 09:25:49 +01:00
samrobinson
64c2406c1e Use TestUtil value for frame count.
PiperOrigin-RevId: 559380901
2023-08-24 09:21:32 +01:00
samrobinson
ff39726368 Remove redundant "aac" tag for dump file.
PiperOrigin-RevId: 559147235
2023-08-24 09:17:32 +01:00
samrobinson
2309fc5edb Add AudioGraph test verifying silence outputs correct amount of bytes.
PiperOrigin-RevId: 559072505
2023-08-22 15:47:54 +01:00
samrobinson
b7782b5119 Inline ProgressHolder and ArgumentCaptor test class variables.
PiperOrigin-RevId: 559069221
2023-08-22 15:46:13 +01:00
samrobinson
7346029f72 Use TemporaryFolder @Rule for automated tear down of temp files.
Reduce @Before specific logic, following java dev guide.

PiperOrigin-RevId: 558825609
2023-08-22 15:42:07 +01:00
samrobinson
ae7667783c Split dump file directories based on input file name.
Remove old unused dump files.

PiperOrigin-RevId: 558820926
2023-08-22 15:39:28 +01:00
huangdarwin
350b394596 Compositor: Add tests for 1, 3, and 5 inputs.
With this, Compositor now handles an arbitrary number of inputs!

PiperOrigin-RevId: 558813361
2023-08-22 15:38:05 +01:00
tofunmi
521c210fd1 Create Timestamp iterator
PiperOrigin-RevId: 558738035
2023-08-22 15:27:42 +01:00
samrobinson
352916b182 Add parameterized tests for generating silence.
Covers all permutations&combinations of:
* AV asset.
* AV asset with effects.
* AV asset with silence.
* AV asset with silence and effects.

Note that the video is not relevant (therefore transmuxed), but is
needed for silence generation.

PiperOrigin-RevId: 558734593
2023-08-22 15:26:19 +01:00
samrobinson
54797d4cc5 Use @Rule and other util to simplify ParameterizedAudioExportTest.
PiperOrigin-RevId: 558168355
2023-08-22 15:19:15 +01:00
samrobinson
99ac6feeee Add varargs convenience constructors to Sequence & Composition.Builder
Improves API ease-of-use & readability and reduces verbose boilerplate.

PiperOrigin-RevId: 558163557
2023-08-22 15:17:55 +01:00
samrobinson
2db6f0aee7 Ensure audio components check incoming data is valid.
Default PCM encoding is only set for decoders outputting raw.

Tests migrated to abide by tighter restrictions.

PiperOrigin-RevId: 558129452
2023-08-18 15:33:44 +01:00
sheenachhabra
15650c6bf3 Add buildUpon method in Composition
We will need to build upon original composition
for pause and resume feature and will have to set modified sequences.
In the next CL, will add method to set sequences on builder and will
probably remove it from the constructor parameter (to avoid providing
two APIs for same thing).

PiperOrigin-RevId: 558127334
2023-08-18 15:32:28 +01:00