whisperbenchmarks/README.md

# WhisperBenchmarks

This repository provides easy-to-use benchmarks using audio and video content from the Internet Archive, specifically targeting various challenging scenarios in audio recordings.

## Links

| Categories            | Title                 | Links                                                                                                 |
|-----------------------|-----------------------|-------------------------------------------------------------------------------------------------------|
| Poor mic placement    | Body camera footage from July 10 traffic stop | [Internet Archive](https://archive.org/details/cobmn-Body_camera_footage_from_July_10_traffic_stop)   |
| Thick accents         | Moonshine for Medicine Popcorn Sutton | [Internet Archive](https://archive.org/details/this-is-the-last-dam-run-of-likker-ill-ever-make-full-movie/+Moonshine+for+Medicine++++Popcorn+Sutton.mp4) |
| Low-quality audio      | 1994 90210 Melrose Place Promos Commercial | [Internet Archive](https://archive.org/details/1994variouscommercials/1994+90210+Melrose+Place+Promos+Commercial.mkv) |
| Artifacts in audio     | 2002 007 Movie Trailer Commercial Bad Video | [Internet Archive](https://archive.org/details/2002variouscommercials/2002+007+Movie+Trailer+Commercial+Bad+Video.mp4) |
| Ideal audio (one speaker) | 8 Bit Bookclub | [Internet Archive](https://archive.org/details/8-bit-bookclub/36+-+ANNOUNCEMENT++SUMMER+HIATUS.mp3) |

## How to Run Whisper Benchmarks

-- TODO --

## Results

Links are embeded for each category

### CPU Benchmarks

|  CPU Model | Poor mic placement (s) | Thick accents (s) | Low-quality audio (s) | Artifacts in audio (s) | Ideal audio (one speaker) | (Docker/Native) |
|-|-|-|-|-|-|-|

### GPU Benchmarks

|  GPU Model | Poor mic placement (s) | Thick accents (s) | Low-quality audio (s) | Artifacts in audio (s) | Ideal audio (one speaker) | (Docker/Native) |
|-|-|-|-|-|-|-|

## Example

## Todo:
- [ ] Write easy bash scripts for running a set of benchmarks with an easy cleanup
- [ ] Finalize a standard format for exporting the data into a spreadsheet
Update README.md 2023-12-02 03:29:27 +00:00			`# WhisperBenchmarks`
Initial commit 2023-12-02 03:16:17 +00:00
Update README.md 2023-12-02 03:29:27 +00:00			`This repository provides easy-to-use benchmarks using audio and video content from the Internet Archive, specifically targeting various challenging scenarios in audio recordings.`

			`## Links`

			`\| Categories \| Title \| Links \|`
			`\|-----------------------\|-----------------------\|-------------------------------------------------------------------------------------------------------\|`
			`\| Poor mic placement \| Body camera footage from July 10 traffic stop \| [Internet Archive](https://archive.org/details/cobmn-Body_camera_footage_from_July_10_traffic_stop) \|`
			`\| Thick accents \| Moonshine for Medicine Popcorn Sutton \| [Internet Archive](https://archive.org/details/this-is-the-last-dam-run-of-likker-ill-ever-make-full-movie/+Moonshine+for+Medicine++++Popcorn+Sutton.mp4) \|`
			`\| Low-quality audio \| 1994 90210 Melrose Place Promos Commercial \| [Internet Archive](https://archive.org/details/1994variouscommercials/1994+90210+Melrose+Place+Promos+Commercial.mkv) \|`
			`\| Artifacts in audio \| 2002 007 Movie Trailer Commercial Bad Video \| [Internet Archive](https://archive.org/details/2002variouscommercials/2002+007+Movie+Trailer+Commercial+Bad+Video.mp4) \|`
Update README.md Add a how-to section and remove example data 2023-12-02 04:06:55 +00:00			`\| Ideal audio (one speaker) \| 8 Bit Bookclub \| [Internet Archive](https://archive.org/details/8-bit-bookclub/36+-+ANNOUNCEMENT++SUMMER+HIATUS.mp3) \|`

			`## How to Run Whisper Benchmarks`

Update README.md 2023-12-02 04:31:39 +00:00			`-- TODO --`
Update README.md 2023-12-02 03:29:27 +00:00
			`## Results`

Update README.md Add a how-to section and remove example data 2023-12-02 04:06:55 +00:00			`Links are embeded for each category`

Update README.md 2023-12-02 03:29:27 +00:00			`### CPU Benchmarks`

Update README.md Add a how-to section and remove example data 2023-12-02 04:06:55 +00:00			`\| CPU Model \| Poor mic placement (s) \| Thick accents (s) \| Low-quality audio (s) \| Artifacts in audio (s) \| Ideal audio (one speaker) \| (Docker/Native) \|`
			`\|-\|-\|-\|-\|-\|-\|-\|`
Update README.md 2023-12-02 03:29:27 +00:00
			`### GPU Benchmarks`

Update README.md Add a how-to section and remove example data 2023-12-02 04:06:55 +00:00			`\| GPU Model \| Poor mic placement (s) \| Thick accents (s) \| Low-quality audio (s) \| Artifacts in audio (s) \| Ideal audio (one speaker) \| (Docker/Native) \|`
			`\|-\|-\|-\|-\|-\|-\|-\|`
Update README.md 2023-12-02 03:29:27 +00:00
Update README.md 2023-12-02 04:31:39 +00:00			`## Example`

Update README.md 2023-12-02 03:29:27 +00:00			`## Todo:`
Update README.md Add a how-to section and remove example data 2023-12-02 04:06:55 +00:00			`- [ ] Write easy bash scripts for running a set of benchmarks with an easy cleanup`
			`- [ ] Finalize a standard format for exporting the data into a spreadsheet`