Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 366 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0578. Min Range: -4.0578

Max Act: 4.0316. Min Act: -0.1700

Data Index: 1867575 (Open Web Text)

Max Activating Token Index: 890

Text #1

Max Range: 4.0578. Min Range: -4.0578

Max Act: 4.0578. Min Act: -0.1700

Data Index: 1867587 (Open Web Text)

Max Activating Token Index: 375

Text #2

Max Range: 4.0578. Min Range: -4.0578

Max Act: 4.0414. Min Act: -0.1700

Data Index: 1495198 (Open Web Text)

Max Activating Token Index: 926

Text #3

Max Range: 4.0578. Min Range: -4.0578

Max Act: 4.0274. Min Act: -0.1700

Data Index: 1495213 (Open Web Text)

Max Activating Token Index: 263

Text #4

Max Range: 4.0578. Min Range: -4.0578

Max Act: 3.9968. Min Act: -0.1700

Data Index: 1495206 (Open Web Text)

Max Activating Token Index: 984

Text #5

Max Range: 4.0578. Min Range: -4.0578

Max Act: 3.9735. Min Act: -0.1700

Data Index: 950457 (Open Web Text)

Max Activating Token Index: 446

Text #6

Max Range: 4.0578. Min Range: -4.0578

Max Act: 3.9693. Min Act: -0.1700

Data Index: 3099355 (Open Web Text)

Max Activating Token Index: 14

Text #7

Max Range: 4.0578. Min Range: -4.0578

Max Act: 3.9586. Min Act: -0.1700

Data Index: 5735557 (Open Web Text)

Max Activating Token Index: 374

Text #8

Max Range: 4.0578. Min Range: -4.0578

Max Act: 3.9485. Min Act: -0.1700

Data Index: 2289732 (Open Web Text)

Max Activating Token Index: 989

Text #9

Max Range: 4.0578. Min Range: -4.0578

Max Act: 3.9571. Min Act: -0.1700

Data Index: 8498217 (Open Web Text)

Max Activating Token Index: 1020

Text #10

Max Range: 4.0578. Min Range: -4.0578

Max Act: 3.9483. Min Act: -0.1700

Data Index: 8063308 (Open Web Text)

Max Activating Token Index: 998

Text #11

Max Range: 4.0578. Min Range: -4.0578

Max Act: 3.9532. Min Act: -0.1700

Data Index: 1998151 (Open Web Text)

Max Activating Token Index: 1018

Text #12

Max Range: 4.0578. Min Range: -4.0578

Max Act: 3.9440. Min Act: -0.1700

Data Index: 7361208 (Open Web Text)

Max Activating Token Index: 837

Text #13

Max Range: 4.0578. Min Range: -4.0578

Max Act: 3.9573. Min Act: -0.1700

Data Index: 7087514 (Open Web Text)

Max Activating Token Index: 731

Text #14

Max Range: 4.0578. Min Range: -4.0578

Max Act: 3.9626. Min Act: -0.1700

Data Index: 6462427 (Open Web Text)

Max Activating Token Index: 18

Text #15

Max Range: 4.0578. Min Range: -4.0578

Max Act: 3.9313. Min Act: -0.1700

Data Index: 4152632 (Open Web Text)

Max Activating Token Index: 301

Text #16

Max Range: 4.0578. Min Range: -4.0578

Max Act: 3.9162. Min Act: -0.1700

Data Index: 4717543 (Open Web Text)

Max Activating Token Index: 1008

Text #17

Max Range: 4.0578. Min Range: -4.0578

Max Act: 3.9356. Min Act: -0.1700

Data Index: 4804798 (Open Web Text)

Max Activating Token Index: 809

Text #18

Max Range: 4.0578. Min Range: -4.0578

Max Act: 3.9105. Min Act: -0.1700

Data Index: 1808048 (Open Web Text)

Max Activating Token Index: 56

Text #19

Max Range: 4.0578. Min Range: -4.0578

Max Act: 3.9265. Min Act: -0.1700

Data Index: 2289714 (Open Web Text)

Max Activating Token Index: 887