Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 386 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.6065. Min Act: -0.1700

Data Index: 4732335 (Open Web Text)

Max Activating Token Index: 386

Text #1

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.6097. Min Act: -0.1700

Data Index: 3876076 (Open Web Text)

Max Activating Token Index: 74

Text #2

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.6016. Min Act: -0.1700

Data Index: 296917 (Open Web Text)

Max Activating Token Index: 264

Text #3

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.5992. Min Act: -0.1700

Data Index: 5429918 (Open Web Text)

Max Activating Token Index: 246

Text #4

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.5827. Min Act: -0.1700

Data Index: 2523338 (Open Web Text)

Max Activating Token Index: 298

Text #5

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.5785. Min Act: -0.1700

Data Index: 6601890 (Open Web Text)

Max Activating Token Index: 22

Text #6

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.5761. Min Act: -0.1700

Data Index: 711216 (Open Web Text)

Max Activating Token Index: 77

Text #7

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.5808. Min Act: -0.1700

Data Index: 3661461 (Open Web Text)

Max Activating Token Index: 359

Text #8

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.5831. Min Act: -0.1700

Data Index: 4988388 (Open Web Text)

Max Activating Token Index: 818

Text #9

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.5784. Min Act: -0.1700

Data Index: 1278672 (Open Web Text)

Max Activating Token Index: 484

Text #10

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.5742. Min Act: -0.1700

Data Index: 4179099 (Open Web Text)

Max Activating Token Index: 92

Text #11

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.5633. Min Act: -0.1700

Data Index: 100286 (Open Web Text)

Max Activating Token Index: 116

Text #12

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.5648. Min Act: -0.1700

Data Index: 6960305 (Open Web Text)

Max Activating Token Index: 631

Text #13

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.5592. Min Act: -0.1700

Data Index: 575919 (Open Web Text)

Max Activating Token Index: 5

Text #14

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.5454. Min Act: -0.1700

Data Index: 858296 (Open Web Text)

Max Activating Token Index: 553

Text #15

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.5479. Min Act: -0.1700

Data Index: 2334150 (Open Web Text)

Max Activating Token Index: 379

Text #16

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.5521. Min Act: -0.1700

Data Index: 5219765 (Open Web Text)

Max Activating Token Index: 250

Text #17

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.5458. Min Act: -0.1700

Data Index: 5332111 (Open Web Text)

Max Activating Token Index: 628

Text #18

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.5477. Min Act: -0.1700

Data Index: 2888799 (Open Web Text)

Max Activating Token Index: 316

Text #19

Max Range: 1.6097. Min Range: -1.6097

Max Act: 1.5363. Min Act: -0.1700

Data Index: 4196727 (Open Web Text)

Max Activating Token Index: 220