Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 602 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.6266. Min Act: -0.1700

Data Index: 131476 (Open Web Text)

Max Activating Token Index: 170

Text #1

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.5597. Min Act: -0.1700

Data Index: 8434721 (Open Web Text)

Max Activating Token Index: 45

Text #2

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.4430. Min Act: -0.1700

Data Index: 2292131 (Open Web Text)

Max Activating Token Index: 431

Text #3

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.3774. Min Act: -0.1700

Data Index: 2586596 (Open Web Text)

Max Activating Token Index: 210

Text #4

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.3697. Min Act: -0.1700

Data Index: 3925921 (Open Web Text)

Max Activating Token Index: 667

Text #5

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.3566. Min Act: -0.1700

Data Index: 7095858 (Open Web Text)

Max Activating Token Index: 23

Text #6

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.3613. Min Act: -0.1700

Data Index: 4852131 (Open Web Text)

Max Activating Token Index: 303

Text #7

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.3429. Min Act: -0.1700

Data Index: 7102358 (Open Web Text)

Max Activating Token Index: 602

Text #8

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.3233. Min Act: -0.1700

Data Index: 6400470 (Open Web Text)

Max Activating Token Index: 246

Text #9

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.2843. Min Act: -0.1700

Data Index: 4132213 (Open Web Text)

Max Activating Token Index: 275

Text #10

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.2504. Min Act: -0.1700

Data Index: 5843918 (Open Web Text)

Max Activating Token Index: 382

Text #11

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.1811. Min Act: -0.1700

Data Index: 1597228 (Open Web Text)

Max Activating Token Index: 878

Text #12

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.1761. Min Act: -0.1700

Data Index: 6978737 (Open Web Text)

Max Activating Token Index: 179

Text #13

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.1453. Min Act: -0.1700

Data Index: 3925012 (Open Web Text)

Max Activating Token Index: 365

Text #14

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.1503. Min Act: -0.1700

Data Index: 8232554 (Open Web Text)

Max Activating Token Index: 856

Text #15

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.1472. Min Act: -0.1700

Data Index: 4827903 (Open Web Text)

Max Activating Token Index: 213

Text #16

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.1239. Min Act: -0.1700

Data Index: 3185832 (Open Web Text)

Max Activating Token Index: 365

Text #17

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.1137. Min Act: -0.1700

Data Index: 6521894 (Open Web Text)

Max Activating Token Index: 313

Text #18

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.1064. Min Act: -0.1700

Data Index: 4646269 (Open Web Text)

Max Activating Token Index: 592

Text #19

Max Range: 4.6266. Min Range: -4.6266

Max Act: 4.1446. Min Act: -0.1700

Data Index: 1594539 (Open Web Text)

Max Activating Token Index: 134