I was greedy and ordered a 60$ capture card from aliexpress, the Ezcap 1080P 60fps Full HD Video Recorder. The recorded output was really disappointing and a big compromise in quality.
I changed course and ordered a Elgato HD60S. Again a slight mistake, because that capture card does not support native UVC (USB Video Class), requiring me to pipe the video content over NDI (video over IP) into OBS.
What I should have bought is the HD60 S+, the one with the plus at the end. For native OBS support without extra steps.
The Elgato HD60S USB capture worked well on windows, after I found the Elgato HD60S windows driver page. The quality of the recorded videos is satisfying for my usecases. Below a comparison of native vs elgato.