Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix example gguf_locally to match chat template requirements #966

Merged
merged 1 commit into from
Dec 3, 2024

Conversation

msk
Copy link
Contributor

@msk msk commented Dec 3, 2024

The gguf_locally example was failing due to a mismatch between the chat template (chat_templates/mistral.json) and the example code. The chat template only supports 'user' and 'assistant' roles, but the example code included a 'system' role message.

When running the example, I encountered the following error:

Error: invalid operation: Conversation roles must alternate user/assistant/user/assistant/... (in chat_template:1)

This PR removes the 'system' role message from the example code to allow it to run successfully.

The chat template used in the gguf_locally example only supports 'user'
and 'assistant' roles, but the example code includes a 'system' role
message. This commit removes the 'system' role message to allow the
example to run successfully.
Copy link

github-actions bot commented Dec 3, 2024

Code Metrics Report
  ===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 C Header                2           35           28            0            7
 Dockerfile              1           41           22           10            9
 JSON                   12          105          104            0            1
 Python                 63         2706         2339           70          297
 Shell                   1           57           22           18           17
 Plain Text              3         3723            0         2413         1310
 TOML                   18          597          531            2           64
 YAML                    2           21           19            2            0
-------------------------------------------------------------------------------
 Jupyter Notebooks       4            0            0            0            0
 |- Markdown             2           77           32           31           14
 |- Python               2          205          178            1           26
 (Total)                            282          210           32           40
-------------------------------------------------------------------------------
 Markdown               43         3312            0         2508          804
 |- BASH                 6          101           98            0            3
 |- JSON                 1           12           12            0            0
 |- Python               7          121          109            0           12
 |- Rust                12          406          344            0           62
 |- TOML                 2           75           63            0           12
 (Total)                           4027          626         2508          893
-------------------------------------------------------------------------------
 Rust                  283        86228        77444         1776         7008
 |- Markdown           137         1482           25         1349          108
 (Total)                          87710        77469         3125         7116
===============================================================================
 Total                 432        96825        80509         6799         9517
===============================================================================
  

@EricLBuehler
Copy link
Owner

Thank you!

@EricLBuehler EricLBuehler merged commit d731ca6 into EricLBuehler:master Dec 3, 2024
12 checks passed
@msk msk deleted the fix-example-gguf-locally branch December 3, 2024 03:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants