Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1452 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.0042. Min Range: -5.0042

Max Act: 5.0042. Min Act: -0.1700

Data Index: 1977380 (Open Web Text)

Max Activating Token Index: 59

Text #1

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.9377. Min Act: -0.1700

Data Index: 4426678 (Open Web Text)

Max Activating Token Index: 733

Text #2

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.9397. Min Act: -0.1700

Data Index: 6605087 (Open Web Text)

Max Activating Token Index: 103

Text #3

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.9455. Min Act: -0.1700

Data Index: 6786492 (Open Web Text)

Max Activating Token Index: 108

Text #4

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.9229. Min Act: -0.1700

Data Index: 404582 (Open Web Text)

Max Activating Token Index: 60

Text #5

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.9420. Min Act: -0.1700

Data Index: 7661790 (Open Web Text)

Max Activating Token Index: 47

Text #6

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.9395. Min Act: -0.1700

Data Index: 4673039 (Open Web Text)

Max Activating Token Index: 450

Text #7

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.8981. Min Act: -0.1700

Data Index: 718134 (Open Web Text)

Max Activating Token Index: 154

Text #8

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.9004. Min Act: -0.1700

Data Index: 6839504 (Open Web Text)

Max Activating Token Index: 508

Text #9

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.9194. Min Act: -0.1700

Data Index: 4659421 (Open Web Text)

Max Activating Token Index: 310

Text #10

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.8927. Min Act: -0.1700

Data Index: 7799478 (Open Web Text)

Max Activating Token Index: 8

Text #11

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.8729. Min Act: -0.1700

Data Index: 6929516 (Open Web Text)

Max Activating Token Index: 628

Text #12

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.8692. Min Act: -0.1700

Data Index: 5090125 (Open Web Text)

Max Activating Token Index: 853

Text #13

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.8720. Min Act: -0.1700

Data Index: 2575031 (Open Web Text)

Max Activating Token Index: 439

Text #14

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.8772. Min Act: -0.1700

Data Index: 2439811 (Open Web Text)

Max Activating Token Index: 257

Text #15

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.8708. Min Act: -0.1700

Data Index: 5498193 (Open Web Text)

Max Activating Token Index: 475

Text #16

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.8877. Min Act: -0.1700

Data Index: 819192 (Open Web Text)

Max Activating Token Index: 17

Text #17

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.8499. Min Act: -0.1700

Data Index: 6284424 (Open Web Text)

Max Activating Token Index: 578

Text #18

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.8281. Min Act: -0.1700

Data Index: 1814421 (Open Web Text)

Max Activating Token Index: 593

Text #19

Max Range: 5.0042. Min Range: -5.0042

Max Act: 4.8595. Min Act: -0.1700

Data Index: 5465665 (Open Web Text)

Max Activating Token Index: 488