update README.md
This commit is contained in:
parent
6713017df9
commit
de9ff79a29
1 changed files with 1 additions and 1 deletions
|
|
@ -250,7 +250,7 @@ def insert_text(rag, file_path):
|
||||||
```
|
```
|
||||||
### Step-2 Generate Queries
|
### Step-2 Generate Queries
|
||||||
|
|
||||||
We extract tokens from both the first half and the second half of each context in the dataset, then combine them to generate queries for dataset descriptions.
|
We extract tokens from both the first half and the second half of each context in the dataset, then combine them as the dataset description to generate queries.
|
||||||
```python
|
```python
|
||||||
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
|
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue