Collabora team has been doing amazing work. Now you don't need to use vendor BSP to use hardware acceleration. Now with upstream linux kernel 7.0, you can play 4K 60fps video with hardware acceleration without any vendor kernel patch.

First of all, thank you so much to my friend for lending me Odroid-M2¹ in this RAMpocalypse.

But how to do it?

I couldn't find a guide on how to actually run it on an upstream kernel. That's why I wrote this article.

I could get some hints from Collabora's blog post². And this Github discussion³ was helpful too.

This is what you need

• Upstream Linux kernel 7.0+
• Enable CONFIG_VIDEO_ROCKCHIP_VDEC=m in the default kernel config.
• linux-firmware (It's for Panthor GPU driver, not for VPU)
• gstreamer 1.28+
• sway or any wayland compositor that supports dmabuf for zero copy (you may be able to use DRM or X11, but not tested)

NOTE:

Some of these aren't needed if you just want decoding. It's done by VPU, so you might only need the upstream kernel. I'm focusing on playing video, which requires integration with GPU. And GPU needs some configurations.

These were enabled by default for me, but you might need to enable them yourself:

• CONFIG_MEDIA_CONTROLLER=y
• CONFIG_MEDIA_CONTROLLER_REQUEST_API=y
• CONFIG_V4L2_MEM2MEM_DEV=y
• CONFIG_DRM_PANTHOR=m

Or you can use ffmpeg/mpv instead of gstreamer. These patches add v4l2request support:

• ffmpeg with this patch⁴ (or you can use this fork⁵)
• mpv with this patch⁶

I believe you can do 'hardware decoding' without any patch with ffmpeg even on TTY. But you may be able to use it only for transcoding or whatever you need it for, but not for playing video AFAIK.

How to play it

v4l2-ctl -d /dev/videoN --list-formats-out
v4l2-ctl -d /dev/videoM --list-formats-out

With patched ffmpeg + mpv on wayland compositor that supports linux-dmabuf-v1:

mpv --hwdec=v4l2request --vo=dmabuf-wayland --hwdec-software-fallback=no bbb_sunflower_2160p_60fps_normal.mp4

--hwdec-software-fallback=no isn't necessary but it's useful for not making it fallback to software decoding.

But how to get board image?

crossdev-stages

You can build image with crossdev-stages. Making an image for RISC-V/ARM boards is usually not that convenient. It removes all the headache when you build an image for boards. It makes a container with hakoniwa and sets up a cross compiler environment with Gentoo Linux. Then you can build rootfs + linux + uboot + whatever automatically, designed so everything can be defined with config files.

I added multi-board support on top of Luca's original work. It used to only support RISC-V boards, now you can use ARM boards too.

Voilà!

Upcoming patches?

There are more works waiting for merge, like 4K 60fps HDMI output. This is really amazing work. You can check details in the resources at the bottom.

Mini VideoBox

I don't want this to just be playing with my Odroid-M2, I want to make it useful.

For future plan, I'm trying to make a 'Mini-VideoBox' someday. FOSDEM has a streaming device called videobox⁷, it's an amazing thing. I had a great chance to use and analyze it, and they also donated one to us, FOSS for All.

But it's bit big and it's for huge conferences. I'm more trying to make a streaming+recording device that can work standalone. Since RK3588 has HDMI input (Unfortunately Odroid-M2 uses RK3588S2 which has no HDMI input), this could be a perfect fit.

My plan is to build several mini-videoboxes for small meetups. I'd love to do this as a FOSS for All project, building it openly with the community. So people can record their talk and meetup painlessly. No OBS on their laptop, no capture card.

We need more time before all features land in the upstream kernel, but I believe it can be done someday.

Thank you!

Kudos to the Collabora team! And again, thanks to my friend C who lent me his Odroid-M2.

Other useful resources

• Mainline status of RK3588
• No Line Like Mainline: Update On The Fully Mainline Software Stack For Rockchip SoCs
• Upstreaming Progress: Video Capture and Camera Support for Recent Rockchip SoCs
• Odroid-M2 wiki
• After FOSDEM VideoBox Updates (BrixIT blog)
• FOSDEM VideoBox 2026 (talk)
• FOSDEM video tooling source

dav1d¹ is an open-source AV1 video decoder that aims for the highest possible performance. The primary goal of this GSoC project was to optimize dav1d's core video processing functions by implementing hand-written assembly using the RISC-V Vector (RVV) Extension. The objective was to maximize performance to enable smooth playback of high-definition AV1 video on low-power RISC-V devices, thereby demonstrating and enhancing the multimedia capabilities of the RISC-V ecosystem.

Key Activities and Achievements

During the project, I performed the following key activities:

• Building a RISC-V Gentoo Development Environment: To facilitate this project, I first built and stabilized a cutting-edge Gentoo Linux development image. This involved using the crossdev-stages² scripts to cross-compile the entire system for RISC-V with full RVV support, a process during which I identified, debugged, and contributed fixes for numerous upstream bugs in core packages like GCC, crossdev, and Perl
• Performance Analysis and Bottleneck Identification: Using the perf tool, I analyzed dav1d's performance to identify bottlenecks. The analysis confirmed that functions like prep_8tap and put_8tap were responsible for the most significant computational load.
• C-based Code Optimization:
  • w_mask C Code Improvement (MR !1804): While implementing the RVV version, I identified an area in the existing C code that could be optimized. By simply pre-calculating and storing frequently used values in variables, I achieved a meaningful ~7% performance improvement on an x86_64 CPU.
• RVV Assembly Optimization: Based on the analysis, I implemented and contributed RVV assembly optimizations for the following core functions. All code was thoroughly tested on Spacemit K1 (VLEN=256) and K230 (VLEN=128) hardware.
  • w_mask RVV Implementation (MR !1797): I implemented RVV assembly for three w_mask functions (444, 422, 420). By applying various techniques such as loop unrolling and dynamic LMUL selection based on VLEN, I achieved a performance increase of up to 16x on Spacemit K1 and up to 9x on K230.
  • emu_edge (MR !1808): I optimized the emu_edge function with RVV, resulting in a performance increase of up to 5x, depending on the input values.

Current Project Status

The submitted Merge Requests have successfully accelerated key dav1d functions using RVV. The optimized code has passed all checkasm and argon conformance tests, ensuring its stability. These changes show significant performance gains across various block widths and on different hardware with VLEN=128 and VLEN=256.

Code Contributions & Merge Requests

The following are the main Merge Requests I worked on and submitted during this GSoC period. You can find detailed code changes, benchmark results, and the review process at each link.

• mc: 8bpc rvv w_mask (v1) (!1797): https://code.videolan.org/videolan/dav1d/-/merge_requests/1797
• mc: 8bpc c w_mask (!1804): https://code.videolan.org/videolan/dav1d/-/merge_requests/1804
• mc: 8bpc rvv emu_edge (!1808): https://code.videolan.org/videolan/dav1d/-/merge_requests/1808

Future Work

While significant progress was made during GSoC, the RISC-V optimization for dav1d is not yet complete. I plan to remain active in the community after GSoC and will continue contributing by addressing the following tasks:

• Optimize prep_8tap and put_8tap: The next goal is to implement RVV assembly for these two functions, which were identified as the biggest bottlenecks by perf.
• Further Optimization: I plan to apply further optimizations, such as eliminating the height-based loop in w_mask for cases where w*h < 64 to process it in a single pass.

Challenges and Key Learnings

Through this project, I gained a deep understanding of RISC-V vector assembly and experienced solving complex performance issues on real hardware. The initial process of understanding RVV was very challenging, but I finally had a breakthrough during a 15-hour flight to Bulgaria, where I could focus intensely on the documentation.

This entire journey would have been impossible without my excellent mentors, Nathan and Luca, and the helpful members of the community. I would like to express my sincere gratitude to everyone who helped me.

Resources

The following resources were extremely helpful throughout the project:

• RISC-V Specifications:
  • RISC-V Vector Specification v1.0
  • RISC-V Bit-Manipulation ISA-Extension
  • RISC-V Assembly Programmer's Cheat Sheet
• RISE Optimization Guide:
  • RISE Project - RISC-V Optimization Guide
• Especially helpful for understanding RVV:
  • RISC-V Vector Extension by EUPILOT
  • A Gentle Introduction to RISC-V Vector Extension by 0x80.pl
  • Optimizing Software for RISC-V (VDD24) by Nathan Egge
  • RISC-V 101 (RISC-V Summit EU) by Nathan Egge

How to build perf

cd linux/tools/perf/pmu-events
./jevents.py riscv spacemit/x60 . pmu-events.c

It will fail, so remove the failing codes in jevents.py.

if len(archs) < 2:
  raise IOError(f'Missing architecture directory \'{_args.arch}\'')

Then, (I know some make options are not needed but it just works).

cd ..
make -j 8 V=1 VF=1 HOSTCC=riscv64-unknown-linux-gnu-gcc HOSTLD=riscv64-unknown-linux-gnu-ld CC=riscv64-unknown-linux-gnu-gcc CXX=riscv64-unknown-linux-gnu-g++ AR=riscv64-unknown-linux-gnu-ar LD=riscv64-unknown-linux-gnu-ld NM=riscv64-unknown-linux-gnu-nm PKG_CONFIG=riscv64-unknown-linux-gnu-pkg-config prefix=/usr bindir_relative=bin tipdir=share/doc/perf-6.8 'EXTRA_CFLAGS=-O2 -pipe' 'EXTRA_LDFLAGS=-Wl,-O1 -Wl,--as-needed' ARCH=riscv BUILD_BPF_SKEL= BUILD_NONDISTRO=1 JDIR= CORESIGHT= GTK2= feature-gtk2-infobar= NO_AUXTRACE= NO_BACKTRACE= NO_DEMANGLE= NO_JEVENTS=0

Note: Cross compile fails, so just build it on your board. It will take around 10mins on Milk-V jupiter.

How to run perf

sudo perf record -e u_mode_cycle ls
sudo perf report # You might not need a root permission

There's other -e options you can try, Read lu_zero's article.¹

perf record --group -e u_mode_cycle,m_mode_cycle,s_mode_cycle
perf record --group -e alu_inst,amo_inst,atomic_inst,fp_div_inst,fp_inst,fp_load_inst,fp_store_inst,load_inst,lr_inst,mult_inst,sc_inst,store_inst,unaligned_load_inst,unaligned_store_inst

What about dav1d?

I used this way:

sudo perf record -e u_mode_cycle dav1d -i sample_video.ivf -o /dev/null

Vendor's Official documentation

I found this document² on Internet.