Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2588 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.4138. Min Range: -6.4138

Max Act: 6.4138. Min Act: -0.1700

Data Index: 3563472 (Open Web Text)

Max Activating Token Index: 109

Text #1

Max Range: 6.4138. Min Range: -6.4138

Max Act: 6.0140. Min Act: -0.1700

Data Index: 3206845 (Open Web Text)

Max Activating Token Index: 852

Text #2

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.8503. Min Act: -0.1700

Data Index: 7654223 (Open Web Text)

Max Activating Token Index: 507

Text #3

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.8289. Min Act: -0.1700

Data Index: 2103385 (Open Web Text)

Max Activating Token Index: 333

Text #4

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.8458. Min Act: -0.1700

Data Index: 1758623 (Open Web Text)

Max Activating Token Index: 389

Text #5

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.8719. Min Act: -0.1700

Data Index: 2394484 (Open Web Text)

Max Activating Token Index: 681

Text #6

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.8283. Min Act: -0.1700

Data Index: 6124232 (Open Web Text)

Max Activating Token Index: 299

Text #7

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.8634. Min Act: -0.1700

Data Index: 5019426 (Open Web Text)

Max Activating Token Index: 400

Text #8

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.8144. Min Act: -0.1700

Data Index: 3347412 (Open Web Text)

Max Activating Token Index: 135

Text #9

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.7598. Min Act: -0.1700

Data Index: 3774940 (Open Web Text)

Max Activating Token Index: 399

Text #10

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.7219. Min Act: -0.1700

Data Index: 6922103 (Open Web Text)

Max Activating Token Index: 313

Text #11

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.7408. Min Act: -0.1700

Data Index: 2324465 (Open Web Text)

Max Activating Token Index: 313

Text #12

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.7270. Min Act: -0.1700

Data Index: 1818313 (Open Web Text)

Max Activating Token Index: 408

Text #13

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.7016. Min Act: -0.1700

Data Index: 6045900 (Open Web Text)

Max Activating Token Index: 146

Text #14

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.6787. Min Act: -0.1700

Data Index: 4065645 (Open Web Text)

Max Activating Token Index: 935

Text #15

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.6510. Min Act: -0.1700

Data Index: 7795167 (Open Web Text)

Max Activating Token Index: 904

Text #16

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.6640. Min Act: -0.1700

Data Index: 1240097 (Open Web Text)

Max Activating Token Index: 894

Text #17

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.6400. Min Act: -0.1700

Data Index: 6992443 (Open Web Text)

Max Activating Token Index: 625

Text #18

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.6792. Min Act: -0.1700

Data Index: 3347426 (Open Web Text)

Max Activating Token Index: 253

Text #19

Max Range: 6.4138. Min Range: -6.4138

Max Act: 5.5859. Min Act: -0.1700

Data Index: 4446729 (Open Web Text)

Max Activating Token Index: 390