Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2888 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.3618. Min Act: -0.1700

Data Index: 1140778 (Open Web Text)

Max Activating Token Index: 245

Text #1

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.2844. Min Act: -0.1700

Data Index: 7131374 (Open Web Text)

Max Activating Token Index: 892

Text #2

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.2222. Min Act: -0.1700

Data Index: 2707356 (Open Web Text)

Max Activating Token Index: 204

Text #3

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.2405. Min Act: -0.1700

Data Index: 1140773 (Open Web Text)

Max Activating Token Index: 762

Text #4

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.2386. Min Act: -0.1700

Data Index: 8075836 (Open Web Text)

Max Activating Token Index: 433

Text #5

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.2144. Min Act: -0.1700

Data Index: 4913202 (Open Web Text)

Max Activating Token Index: 539

Text #6

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.1982. Min Act: -0.1700

Data Index: 1631810 (Open Web Text)

Max Activating Token Index: 596

Text #7

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.2247. Min Act: -0.1700

Data Index: 2707357 (Open Web Text)

Max Activating Token Index: 203

Text #8

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.2136. Min Act: -0.1700

Data Index: 3259013 (Open Web Text)

Max Activating Token Index: 594

Text #9

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.2204. Min Act: -0.1700

Data Index: 7326589 (Open Web Text)

Max Activating Token Index: 272

Text #10

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.1863. Min Act: -0.1700

Data Index: 8826914 (Open Web Text)

Max Activating Token Index: 409

Text #11

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.1731. Min Act: -0.1700

Data Index: 8764630 (Open Web Text)

Max Activating Token Index: 425

Text #12

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.1835. Min Act: -0.1700

Data Index: 7617481 (Open Web Text)

Max Activating Token Index: 809

Text #13

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.1505. Min Act: -0.1700

Data Index: 529974 (Open Web Text)

Max Activating Token Index: 303

Text #14

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.1643. Min Act: -0.1700

Data Index: 1980154 (Open Web Text)

Max Activating Token Index: 610

Text #15

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.1843. Min Act: -0.1700

Data Index: 6894073 (Open Web Text)

Max Activating Token Index: 842

Text #16

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.1340. Min Act: -0.1700

Data Index: 8331403 (Open Web Text)

Max Activating Token Index: 416

Text #17

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.1500. Min Act: -0.1700

Data Index: 7617480 (Open Web Text)

Max Activating Token Index: 484

Text #18

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.1807. Min Act: -0.1700

Data Index: 6435133 (Open Web Text)

Max Activating Token Index: 229

Text #19

Max Range: 3.3618. Min Range: -3.3618

Max Act: 3.1307. Min Act: -0.1700

Data Index: 626374 (Open Web Text)

Max Activating Token Index: 472