This is more clear than using Format.NO_VALUE, when we do actually intend for an
output value.
Also, fix @see formatting by using summary fragments instead, and add an error
output for OETF and EOTF transfer functions.
PiperOrigin-RevId: 490910229
Previously, FrameProcessor never had the usecase in which the output surface
is replaced, while previewing introduced this usecase.
When switching output surfaces, we need to destroy the EGL Surface linked to the
surface that is being swapped out, because an EGL surface is linked to the EGL
display (which is not destroyed even when releasing FrameProcessor).
A GL exception will be thrown in the following scenario if we don't destroy the
EGL surface:
1. Creates a Surface, the surface is identified by address 0x11
2. Sets Surface(0x11) on FrameProcessor. Eventually an EGL surface is created
to wrap Surface(0x11)
3. Release FrameProcess, this releases the EGL context
4. Instantiate a new FrameProcessor, sets Surface(0x11) as the output
5. When FrameProcessor creates an EGL surface to wrap Surface(0x11), GL throws
an exception, becasue Surface(0x11) has previouly been connected to an EGL
surface.
PiperOrigin-RevId: 489590072
`availableFrameCount` tracks the number of frames that is avilable on the
`SurfaceTexture`, but haven't been used (by `updateTexImage()`) yet. Thus
semantically this counter should only be decremented after calling
`updateTexImage()`, not before it.
Also reworded `getPendingFrameCount()` javadoc, "external texture" is an
internal state that is not publicised anywhere.
PiperOrigin-RevId: 488765174
Public methods may only refer to public types in their signature. This
change ensures that by switching to a public supertype everywhere.
PiperOrigin-RevId: 485568625
Make it easier to support use of RGBA_101012 rather than RGBA_8888 for EGL
contexts, displays, and surfaces.
This tangentially supports adding HDR tests, by slightly simplifying the color
selection logic we'd have to add in HDR tests.
PiperOrigin-RevId: 482219428
Before, they used `width` and `height`, which was inconsistent with other pixel tests, and less descriptive.
Refactoring change only. No functional change intended.
PiperOrigin-RevId: 481970243
Currently, a frame is dropped if it's requested release time is in the past.
This mode was added to support previewing. However, in normal ExoPlayer
playback, slightly late frames (<30ms late) are also rendered. On MediaCodec
side, this means calling `releaseOutputBuffer` with a release time in the
past.
PiperOrigin-RevId: 479615291
This mode is supported by using `C.TIME_UNSET` (which is a negative value). The
new logic decouples the value of `C.TIME_UNSET` and the frame dropping
behaviour.
PiperOrigin-RevId: 479368880
Currently `FrameProcessor.releaseOutputFrame()` method supports
Release at a specific system time
Drops the frame
This API is not that convenient to use when the caller wants to release a frame, now, regardless of the release time. A use case is to release (present) a frame when no frame is shown for a while, and it's thus better to just release the frame, now.
Currently if MCVR wants a frame to be rendered now, MCVR calls release frame with a set offset like 10us: `releaseOutputFrame(System.nanoTime() + 10_000)`. The 10us offset is to prevent the frame processor dropping the frame, due to thread hopping delays.
To make the API better usable, consider adding a mode for releasing the frame now, like (bold marks the new mode)
- Use C.TIME_UNSET to drop
- **Use -1 to release the frame immediately, or**
- Use an actual release time.
PiperOrigin-RevId: 479044215
Rename test files to avoid substrings that can be implied by the directory name,
like "Transformation" and "Test"
No functional changes. Renaming-only.
PiperOrigin-RevId: 477724724
* Before this CL, the texture was stored during the construction of the LUT processor. This failed since if one creates a list of GlEffects on the application thread, the texture will get stored in the application thread during the effect creation and not on the GL thread, which executes the FrameProcessors.
* This is an issue since the executing thread then can't index from the texture stored on a different thread.
PiperOrigin-RevId: 476388021
* Transform the intermediate color space to linear SDR by applying the SMPTE 170M EOTF and OETF.
* Use linear colors for the color filter pixel tests and update all golden bitmaps.
PiperOrigin-RevId: 476124592
TL;DR: we should check if there are new frames available to queue to the
ExternalTextureProcessor before actually queueing a frame.
The overall flow on the external texture processor:
- `SurfaceTexture.onFrameAvailable` is called on `ExtTexMgr`, and
- it calls `updateTexImage()`, and sets `frame`
- it calls `maybeQueueFrameToExtTexProc()`
- the frame is queued to `ExtTexProc` if `frame` is set
- From `ExtTexProc.queueInputFrame()`:
- notifies the `frameProcessorListener` of available frame
- notifies the `inputListener` of `onReadyToAcceptInputFrame`
- (`ExtTexMgr` is the listener), it calls `maybeQueueFrameToExtTexProc()`
again
-- Parallelly --
- `ExtTexProc` calls `inputListener.onInputFrameProcessed`, when the frame is
released
- (`ExtTexMgr` is the listener), sets `frame` to `null`
*Problem*
This logic relies on `frame` to be cleared at the right time.
In transformer, it's OK b/c `ExtTexProc` release the frame immediately in
`queueInputFrame()` and calls `onInputFrameProcessed` which also reset `frame`
But in previewing, the frame is not released for a while, up to 10 ms.
In this case, `frame` will not reset in this 10 ms, and
`maybeQueueFrameToExtTexProc()` is repeatedly queueing the same input frame.
PiperOrigin-RevId: 470211620
Use the PQ OETF and EOTF to ensure that intermediate fragment shader operations
using PQ are in linear BT.2020 rather than PQ and HLG-1 BT.2020.
Also, swap the OETF and EOTF in shaders, as they were used incorrectly before
Manually tested by verifying transformer demo HLG and PQ videos look the same with and without this CL, including with a BitmapOverlayProcessor enabled to test flows both with one MatrixTransformationProcessor that skips HDR TFs, and with one that doesn't.
PiperOrigin-RevId: 469736067
Adds a method to FrameProcessor.Listener to be called when an
output frame is available and a method releaseOutputFrame in
FrameProcessor allowing the caller to trigger release of the
oldest available output frame at a given timestamp. Late frames
or frames with unset release times are dropped in the
FinalMatrixTransformationProcessorWrapper.
More than one output frame can become available before they are
released if the penultimate GlTextureProcessor is capable of producing
multiple output frames. Processing continues while waiting for
releaseOutputFrame to be called. Frame release tasks are prioritized
over other tasks.
PiperOrigin-RevId: 468473072
Manually tested using transformer demo HLG videos. Before this CL, RGB values after the YUV to RGB conversion reached up to 1.025. After this CL, RGB values correctly clamp at 1.0.
PiperOrigin-RevId: 468426092
FrameProcessingTaskExecutor should be released on error.
There can be a delay until this happens, so
FrameProcessingTaskExecutor will cancel any pending tasks
and drop new tasks until it is released.
PiperOrigin-RevId: 468171820
This is needed as a pre-requisite for allowing MCVR to control
FrameProcessor frame release for previewing.
Submitting a high-priority task is conceptually different from
posting at the front of a single queue of tasks, as the high-priority
tasks are executed in FIFO order among themselves. This will ensure
that frame release tasks submitted in close succession are executed
in the order they are submitted but before any lower priority tasks.
PiperOrigin-RevId: 467675137
