Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2565 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.3977. Min Act: -0.1700

Data Index: 4665991 (Open Web Text)

Max Activating Token Index: 430

Text #1

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.4109. Min Act: -0.1700

Data Index: 5962122 (Open Web Text)

Max Activating Token Index: 768

Text #2

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.3775. Min Act: -0.1700

Data Index: 479333 (Open Web Text)

Max Activating Token Index: 394

Text #3

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.3027. Min Act: -0.1700

Data Index: 4026531 (Open Web Text)

Max Activating Token Index: 905

Text #4

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.2850. Min Act: -0.1700

Data Index: 4379517 (Open Web Text)

Max Activating Token Index: 519

Text #5

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.3076. Min Act: -0.1700

Data Index: 8002198 (Open Web Text)

Max Activating Token Index: 15

Text #6

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.2873. Min Act: -0.1700

Data Index: 8173524 (Open Web Text)

Max Activating Token Index: 532

Text #7

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.3074. Min Act: -0.1700

Data Index: 3984466 (Open Web Text)

Max Activating Token Index: 719

Text #8

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.2928. Min Act: -0.1700

Data Index: 1281259 (Open Web Text)

Max Activating Token Index: 671

Text #9

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.2594. Min Act: -0.1700

Data Index: 1235436 (Open Web Text)

Max Activating Token Index: 73

Text #10

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.2863. Min Act: -0.1700

Data Index: 2346692 (Open Web Text)

Max Activating Token Index: 173

Text #11

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.2612. Min Act: -0.1700

Data Index: 7049705 (Open Web Text)

Max Activating Token Index: 43

Text #12

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.2515. Min Act: -0.1700

Data Index: 6175731 (Open Web Text)

Max Activating Token Index: 160

Text #13

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.2430. Min Act: -0.1700

Data Index: 5961587 (Open Web Text)

Max Activating Token Index: 538

Text #14

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.2633. Min Act: -0.1700

Data Index: 7433604 (Open Web Text)

Max Activating Token Index: 731

Text #15

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.2569. Min Act: -0.1700

Data Index: 6441661 (Open Web Text)

Max Activating Token Index: 402

Text #16

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.2399. Min Act: -0.1700

Data Index: 2231862 (Open Web Text)

Max Activating Token Index: 797

Text #17

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.2646. Min Act: -0.1700

Data Index: 8578047 (Open Web Text)

Max Activating Token Index: 212

Text #18

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.2429. Min Act: -0.1700

Data Index: 7333046 (Open Web Text)

Max Activating Token Index: 380

Text #19

Max Range: 2.4109. Min Range: -2.4109

Max Act: 2.2264. Min Act: -0.1700

Data Index: 4290023 (Open Web Text)

Max Activating Token Index: 173