Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bench: add dataset param for bench_multiturn #3990

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

zeroorhero
Copy link

@zeroorhero zeroorhero commented Mar 2, 2025

In bench_multiturn.py file, the dataset is downloaded from HuggingFace by default, which may sometimes fail. Therefore, a new parameter is added to specify the path to the local dataset.

Motivation

Modifications

Checklist

@zeroorhero
Copy link
Author

cc @xiezhq-hermann

@xiezhq-hermann
Copy link
Collaborator

hi @zeroorhero thank you for your interest in using the benchmark. But I do not think it is a necessary nor a very good change, as if the default option is set to a local file it is even easier to fail for users without it in hand.

@zeroorhero
Copy link
Author

hi @zeroorhero thank you for your interest in using the benchmark. But I do not think it is a necessary nor a very good change, as if the default option is set to a local file it is even easier to fail for users without it in hand.
1111
In bench_serving.py, if the default path does not exist, it will proceed to download the dataset. Perhaps setting the default value to an empty string ("") is a good choice, and I will make this change later. However, in China, this option is still very much needed because it is often difficult to successfully download from Hugging Face.

In bench_multiturn.py file, the dataset is
downloaded from HuggingFace by default,
which may sometimes fail. Therefore, a new
parameter is added to specify the path to
the local dataset.

Signed-off-by: Changqi Lu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants