whisperbenchmarks/README.md

# WhisperBenchmarks

This repository provides easy-to-use benchmarks using audio and video content from the Internet Archive, specifically targeting various challenging scenarios in audio recordings.

## Links

| Categories            | Title                 | Links                                                                                                 |
|-----------------------|-----------------------|-------------------------------------------------------------------------------------------------------|
| Poor mic placement    | Body camera footage from July 10 traffic stop | [Internet Archive](https://archive.org/details/cobmn-Body_camera_footage_from_July_10_traffic_stop)   |
| Thick accents         | Moonshine for Medicine Popcorn Sutton | [Internet Archive](https://archive.org/details/this-is-the-last-dam-run-of-likker-ill-ever-make-full-movie/+Moonshine+for+Medicine++++Popcorn+Sutton.mp4) |
| Low-quality audio      | 1994 90210 Melrose Place Promos Commercial | [Internet Archive](https://archive.org/details/1994variouscommercials/1994+90210+Melrose+Place+Promos+Commercial.mkv) |
| Artifacts in audio     | 2002 007 Movie Trailer Commercial Bad Video | [Internet Archive](https://archive.org/details/2002variouscommercials/2002+007+Movie+Trailer+Commercial+Bad+Video.mp4) |

## Results

### CPU Benchmarks

| Video                                                      | Time (s) | CPU Model                         |
|------------------------------------------------------------|----------|-----------------------------------|
| [Body camera footage from July 10 traffic stop](https://archive.org/details/cobmn-Body_camera_footage_from_July_10_traffic_stop)    | 15.25    | Intel Core i9-9900K               |
| [Moonshine for Medicine Popcorn Sutton ](https://archive.org/details/this-is-the-last-dam-run-of-likker-ill-ever-make-full-movie/+Moonshine+for+Medicine++++Popcorn+Sutton.mp4)   | 22.50    | AMD Ryzen 7 5800X                 |
| [1994 90210 Melrose Place Promos Commercial](https://archive.org/details/1994variouscommercials/1994+90210+Melrose+Place+Promos+Commercial.mkv) | 10.75    | Intel Core i7-10700K              |
| [2002 007 Movie Trailer Commercial Bad Video](https://archive.org/details/2002variouscommercials/2002+007+Movie+Trailer+Commercial+Bad+Video.mp4)     | 18.30    | AMD Ryzen 9 5900X                 |

### GPU Benchmarks

| Video                                                      | Time (s) | GPU Model                         |
|------------------------------------------------------------|----------|-----------------------------------|
| [Body camera footage from July 10 traffic stop](https://archive.org/details/cobmn-Body_camera_footage_from_July_10_traffic_stop)    | 8.50     | NVIDIA GeForce RTX 3080           |
| [Moonshine for Medicine Popcorn Sutton ](https://archive.org/details/this-is-the-last-dam-run-of-likker-ill-ever-make-full-movie/+Moonshine+for+Medicine++++Popcorn+Sutton.mp4)   | 15.75    | NVIDIA Quadro P5000              |
| [1994 90210 Melrose Place Promos Commercial](https://archive.org/details/1994variouscommercials/1994+90210+Melrose+Place+Promos+Commercial.mkv) | 8.20     | AMD Radeon RX 6800 XT            |
| [2002 007 Movie Trailer Commercial Bad Video](https://archive.org/details/2002variouscommercials/2002+007+Movie+Trailer+Commercial+Bad+Video.mp4)     | 12.45    | NVIDIA Tesla V100                |


## Todo:
- [ ] Write easy bash scripts for running a set of benchmarks
- [ ] Create a standard format for exporting the data into a spreadsheet