Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2197 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.5953. Min Act: -0.1700

Data Index: 4016429 (Open Web Text)

Max Activating Token Index: 849

Text #1

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.3811. Min Act: -0.1700

Data Index: 5686272 (Open Web Text)

Max Activating Token Index: 714

Text #2

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.3750. Min Act: -0.1700

Data Index: 3009332 (Open Web Text)

Max Activating Token Index: 535

Text #3

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.3116. Min Act: -0.1700

Data Index: 7288039 (Open Web Text)

Max Activating Token Index: 180

Text #4

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.2628. Min Act: -0.1700

Data Index: 3864885 (Open Web Text)

Max Activating Token Index: 329

Text #5

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.2832. Min Act: -0.1700

Data Index: 388937 (Open Web Text)

Max Activating Token Index: 498

Text #6

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.2338. Min Act: -0.1700

Data Index: 7615512 (Open Web Text)

Max Activating Token Index: 713

Text #7

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.2217. Min Act: -0.1700

Data Index: 4648868 (Open Web Text)

Max Activating Token Index: 599

Text #8

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.2055. Min Act: -0.1700

Data Index: 138942 (Open Web Text)

Max Activating Token Index: 305

Text #9

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.1940. Min Act: -0.1700

Data Index: 2944256 (Open Web Text)

Max Activating Token Index: 659

Text #10

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.1815. Min Act: -0.1700

Data Index: 7751442 (Open Web Text)

Max Activating Token Index: 153

Text #11

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.1708. Min Act: -0.1700

Data Index: 443349 (Open Web Text)

Max Activating Token Index: 524

Text #12

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.1513. Min Act: -0.1700

Data Index: 6205711 (Open Web Text)

Max Activating Token Index: 477

Text #13

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.1386. Min Act: -0.1700

Data Index: 2182474 (Open Web Text)

Max Activating Token Index: 302

Text #14

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.1415. Min Act: -0.1700

Data Index: 4429497 (Open Web Text)

Max Activating Token Index: 215

Text #15

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.1346. Min Act: -0.1700

Data Index: 4340760 (Open Web Text)

Max Activating Token Index: 759

Text #16

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.1412. Min Act: -0.1700

Data Index: 6324961 (Open Web Text)

Max Activating Token Index: 109

Text #17

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.1344. Min Act: -0.1700

Data Index: 8665507 (Open Web Text)

Max Activating Token Index: 623

Text #18

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.1251. Min Act: -0.1700

Data Index: 4046490 (Open Web Text)

Max Activating Token Index: 482

Text #19

Max Range: 3.5953. Min Range: -3.5953

Max Act: 3.1226. Min Act: -0.1700

Data Index: 1013090 (Open Web Text)

Max Activating Token Index: 568