Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 432 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.7174. Min Act: -0.1700

Data Index: 98036 (Open Web Text)

Max Activating Token Index: 477

Text #1

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.7033. Min Act: -0.1700

Data Index: 5480222 (Open Web Text)

Max Activating Token Index: 475

Text #2

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.7105. Min Act: -0.1700

Data Index: 2699829 (Open Web Text)

Max Activating Token Index: 483

Text #3

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.4204. Min Act: -0.1700

Data Index: 8811944 (Open Web Text)

Max Activating Token Index: 1008

Text #4

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.3652. Min Act: -0.1700

Data Index: 4406970 (Open Web Text)

Max Activating Token Index: 492

Text #5

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.3032. Min Act: -0.1700

Data Index: 1325262 (Open Web Text)

Max Activating Token Index: 990

Text #6

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.2822. Min Act: -0.1700

Data Index: 3477130 (Open Web Text)

Max Activating Token Index: 519

Text #7

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.1842. Min Act: -0.1700

Data Index: 6105727 (Open Web Text)

Max Activating Token Index: 1003

Text #8

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.2484. Min Act: -0.1700

Data Index: 7896579 (Open Web Text)

Max Activating Token Index: 992

Text #9

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.2481. Min Act: -0.1700

Data Index: 4773243 (Open Web Text)

Max Activating Token Index: 503

Text #10

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.2155. Min Act: -0.1700

Data Index: 350652 (Open Web Text)

Max Activating Token Index: 444

Text #11

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.1467. Min Act: -0.1700

Data Index: 6035157 (Open Web Text)

Max Activating Token Index: 471

Text #12

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.1328. Min Act: -0.1700

Data Index: 1620398 (Open Web Text)

Max Activating Token Index: 989

Text #13

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.1187. Min Act: -0.1700

Data Index: 7353953 (Open Web Text)

Max Activating Token Index: 509

Text #14

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.1095. Min Act: -0.1700

Data Index: 472987 (Open Web Text)

Max Activating Token Index: 523

Text #15

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.1302. Min Act: -0.1700

Data Index: 8006842 (Open Web Text)

Max Activating Token Index: 479

Text #16

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.0420. Min Act: -0.1700

Data Index: 4776031 (Open Web Text)

Max Activating Token Index: 528

Text #17

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.0967. Min Act: -0.1700

Data Index: 5417754 (Open Web Text)

Max Activating Token Index: 482

Text #18

Max Range: 6.7174. Min Range: -6.7174

Max Act: 6.0844. Min Act: -0.1700

Data Index: 8532366 (Open Web Text)

Max Activating Token Index: 988

Text #19

Max Range: 6.7174. Min Range: -6.7174

Max Act: 5.9739. Min Act: -0.1700

Data Index: 192382 (Open Web Text)

Max Activating Token Index: 1023