Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1773 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.4569. Min Act: -0.1700

Data Index: 3318104 (Open Web Text)

Max Activating Token Index: 239

Text #1

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.3049. Min Act: -0.1700

Data Index: 5628208 (Open Web Text)

Max Activating Token Index: 402

Text #2

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.3106. Min Act: -0.1700

Data Index: 6301888 (Open Web Text)

Max Activating Token Index: 342

Text #3

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.2615. Min Act: -0.1700

Data Index: 1609291 (Open Web Text)

Max Activating Token Index: 653

Text #4

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.2715. Min Act: -0.1700

Data Index: 8459319 (Open Web Text)

Max Activating Token Index: 589

Text #5

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.1970. Min Act: -0.1700

Data Index: 6535202 (Open Web Text)

Max Activating Token Index: 502

Text #6

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.1730. Min Act: -0.1700

Data Index: 4633002 (Open Web Text)

Max Activating Token Index: 506

Text #7

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.1185. Min Act: -0.1700

Data Index: 5637531 (Open Web Text)

Max Activating Token Index: 546

Text #8

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.1754. Min Act: -0.1700

Data Index: 4252231 (Open Web Text)

Max Activating Token Index: 539

Text #9

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.1762. Min Act: -0.1700

Data Index: 5730993 (Open Web Text)

Max Activating Token Index: 269

Text #10

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.1525. Min Act: -0.1700

Data Index: 5061133 (Open Web Text)

Max Activating Token Index: 125

Text #11

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.1670. Min Act: -0.1700

Data Index: 1318297 (Open Web Text)

Max Activating Token Index: 490

Text #12

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.1354. Min Act: -0.1700

Data Index: 7051566 (Open Web Text)

Max Activating Token Index: 440

Text #13

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.1359. Min Act: -0.1700

Data Index: 7698005 (Open Web Text)

Max Activating Token Index: 361

Text #14

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.1009. Min Act: -0.1700

Data Index: 6022342 (Open Web Text)

Max Activating Token Index: 805

Text #15

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.0902. Min Act: -0.1700

Data Index: 5145146 (Open Web Text)

Max Activating Token Index: 849

Text #16

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.0864. Min Act: -0.1700

Data Index: 5183174 (Open Web Text)

Max Activating Token Index: 217

Text #17

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.0699. Min Act: -0.1700

Data Index: 8052694 (Open Web Text)

Max Activating Token Index: 396

Text #18

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.0701. Min Act: -0.1700

Data Index: 5533788 (Open Web Text)

Max Activating Token Index: 378

Text #19

Max Range: 5.4569. Min Range: -5.4569

Max Act: 5.0356. Min Act: -0.1700

Data Index: 237366 (Open Web Text)

Max Activating Token Index: 708