Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 130 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.6811. Min Act: -0.1657

Data Index: 8805004 (Open Web Text)

Max Activating Token Index: 11

Text #1

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5766. Min Act: -0.1694

Data Index: 5601343 (Open Web Text)

Max Activating Token Index: 556

Text #2

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5536. Min Act: -0.1605

Data Index: 1583542 (Open Web Text)

Max Activating Token Index: 127

Text #3

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5488. Min Act: -0.1670

Data Index: 2398078 (Open Web Text)

Max Activating Token Index: 839

Text #4

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5465. Min Act: -0.1473

Data Index: 1411426 (Open Web Text)

Max Activating Token Index: 210

Text #5

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5347. Min Act: -0.1697

Data Index: 4551285 (Open Web Text)

Max Activating Token Index: 463

Text #6

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5243. Min Act: -0.1690

Data Index: 1874317 (Open Web Text)

Max Activating Token Index: 16

Text #7

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5302. Min Act: -0.1681

Data Index: 1269193 (Open Web Text)

Max Activating Token Index: 399

Text #8

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5214. Min Act: -0.1540

Data Index: 1010389 (Open Web Text)

Max Activating Token Index: 498

Text #9

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5218. Min Act: -0.1696

Data Index: 4640567 (Open Web Text)

Max Activating Token Index: 757

Text #10

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5143. Min Act: -0.1668

Data Index: 8636513 (Open Web Text)

Max Activating Token Index: 331

Text #11

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5192. Min Act: -0.1517

Data Index: 8394545 (Open Web Text)

Max Activating Token Index: 376

Text #12

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5166. Min Act: -0.1667

Data Index: 6270019 (Open Web Text)

Max Activating Token Index: 5

Text #13

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5277. Min Act: -0.1481

Data Index: 7616363 (Open Web Text)

Max Activating Token Index: 237

Text #14

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5244. Min Act: -0.1698

Data Index: 7801902 (Open Web Text)

Max Activating Token Index: 800

Text #15

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5188. Min Act: -0.1697

Data Index: 1381249 (Open Web Text)

Max Activating Token Index: 316

Text #16

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5172. Min Act: -0.1628

Data Index: 7057931 (Open Web Text)

Max Activating Token Index: 672

Text #17

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5085. Min Act: -0.1687

Data Index: 329149 (Open Web Text)

Max Activating Token Index: 911

Text #18

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5122. Min Act: -0.1700

Data Index: 600473 (Open Web Text)

Max Activating Token Index: 341

Text #19

Max Range: 1.6811. Min Range: -1.6811

Max Act: 1.5114. Min Act: -0.1699

Data Index: 7992376 (Open Web Text)

Max Activating Token Index: 550