update README.md

This commit is contained in:
LarFii 2024-10-11 15:19:20 +08:00
parent c630f4e96c
commit 38149a2130

View file

@ -250,7 +250,7 @@ def insert_text(rag, file_path):
```
### Step-2 Generate Queries
We extract tokens from both the first half and the second half of each context in the dataset, then combine them to generate queries for dataset descriptions.
We extract tokens from both the first half and the second half of each context in the dataset, then combine them as the dataset description to generate queries.
```python
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')