brooke/whisperbenchmarks
1
0
Fork 0
Code Issues Wiki Activity

update

Browse source
This commit is contained in:
rouce 2023-12-03 00:03:32 -05:00
parent edaa5321a1
commit c9946e316b
1 changed files with 3 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
README.md
View file

@ -42,9 +42,10 @@ Results are for the complete run which includes loading the model, running VAD,
| GPU Model | Poor mic placement (m:s:ms) | Thick accents (m:s:ms) | Artifacts in audio (m:s:ms) | Ideal audio (m:s:ms) | Long form (m:s:ms) | (Docker/Native) | Model | | GPU Model | Poor mic placement (m:s:ms) | Thick accents (m:s:ms) | Artifacts in audio (m:s:ms) | Ideal audio (m:s:ms) | Long form (m:s:ms) | (Docker/Native) | Model |
|-|-|-|-|-|-|-|-| |-|-|-|-|-|-|-|-|
| RTX 2060S | [00:11:12](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/2060S/Body_camera_footage_from_July_10_traffic_stop.mp4-subs.srt) | [00:06:96](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/2060S/Moonshine%20for%20Medicine%20Popcorn%20Sutton.mp4-subs.srt) | [00:04:40](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/2060S/2002%20007%20Movie%20Trailer%20Commercial%20Bad%20Video.mp4-subs.srt) | [00:08:31](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/2060S/36%20-%20ANNOUNCEMENT%20SUMMER%20HIATUS.mp3-subs.srt) | 03:34:00 | Native | faster-medium | | RTX 2060S | [00:11:12](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/2060S/Body_camera_footage_from_July_10_traffic_stop.mp4-subs.srt) | [00:06:96](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/2060S/Moonshine%20for%20Medicine%20Popcorn%20Sutton.mp4-subs.srt) | [00:04:40](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/2060S/2002%20007%20Movie%20Trailer%20Commercial%20Bad%20Video.mp4-subs.srt) | [00:08:31](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/2060S/36%20-%20ANNOUNCEMENT%20SUMMER%20HIATUS.mp3-subs.srt) | [03:34:00](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/2060S/Black%20Magic-NA_x264.mp4-subs.srt) | Native | faster-medium |
| RTX A5000 | [00:10:86](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/A5000/Body_camera_footage_from_July_10_traffic_stop.mp4-subs.srt) | [00:07.84](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/A5000/Moonshine%20for%20Medicine%20Popcorn%20Sutton.mp4-subs.srt) | [00:07:75](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/A5000/2002%20007%20Movie%20Trailer%20Commercial%20Bad%20Video.mp4-subs.srt) | [00:08:91](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/A5000/36%20-%20ANNOUNCEMENT%20SUMMER%20HIATUS.mp3-subs.srt) | 03:17:10 | Native | faster-large-v3 | | RTX A5000 | [00:10:86](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/A5000/Body_camera_footage_from_July_10_traffic_stop.mp4-subs.srt) | [00:07.84](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/A5000/Moonshine%20for%20Medicine%20Popcorn%20Sutton.mp4-subs.srt) | [00:07:75](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/A5000/2002%20007%20Movie%20Trailer%20Commercial%20Bad%20Video.mp4-subs.srt) | [00:08:91](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/A5000/36%20-%20ANNOUNCEMENT%20SUMMER%20HIATUS.mp3-subs.srt) | [03:17:10](https://git.myco.systems/brooke/whisperbenchmarks/src/branch/main/benchmark-outputs/A5000/Black%20Magic-NA_x264.mp4-subs.srt) | Native | faster-large-v3 |
## Todo: ## Todo:
- [ ] Write easy bash scripts for running a set of benchmarks with an easy cleanup - [ ] Write easy bash scripts for running a set of benchmarks with an easy cleanup
- [ ] Finalize a standard format for exporting the data into a spreadsheet - [ ] Finalize a standard format for exporting the data into a spreadsheet
- [ ] Some outputs seem to be presenting artifacts when using faster-whisper, there should be some kind of easy review process so that batches can quickly be re-run, maybe a link to open vlc with the new subtitles loaded?