Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2503 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.9288. Min Act: -0.1700

Data Index: 7082635 (Open Web Text)

Max Activating Token Index: 754

Text #1

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.8939. Min Act: -0.1700

Data Index: 3710824 (Open Web Text)

Max Activating Token Index: 708

Text #2

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.7073. Min Act: -0.1700

Data Index: 6754347 (Open Web Text)

Max Activating Token Index: 1001

Text #3

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.6831. Min Act: -0.1700

Data Index: 5458941 (Open Web Text)

Max Activating Token Index: 556

Text #4

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.6818. Min Act: -0.1700

Data Index: 5724224 (Open Web Text)

Max Activating Token Index: 96

Text #5

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.6966. Min Act: -0.1700

Data Index: 8197391 (Open Web Text)

Max Activating Token Index: 37

Text #6

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.6865. Min Act: -0.1700

Data Index: 973281 (Open Web Text)

Max Activating Token Index: 180

Text #7

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.6852. Min Act: -0.1700

Data Index: 6754348 (Open Web Text)

Max Activating Token Index: 945

Text #8

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.6920. Min Act: -0.1700

Data Index: 2201592 (Open Web Text)

Max Activating Token Index: 952

Text #9

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.6707. Min Act: -0.1700

Data Index: 8777866 (Open Web Text)

Max Activating Token Index: 933

Text #10

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.5951. Min Act: -0.1700

Data Index: 8714914 (Open Web Text)

Max Activating Token Index: 683

Text #11

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.6378. Min Act: -0.1700

Data Index: 400973 (Open Web Text)

Max Activating Token Index: 871

Text #12

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.6400. Min Act: -0.1700

Data Index: 3870143 (Open Web Text)

Max Activating Token Index: 685

Text #13

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.6335. Min Act: -0.1700

Data Index: 3298906 (Open Web Text)

Max Activating Token Index: 29

Text #14

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.6425. Min Act: -0.1700

Data Index: 1854131 (Open Web Text)

Max Activating Token Index: 114

Text #15

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.6532. Min Act: -0.1700

Data Index: 1848720 (Open Web Text)

Max Activating Token Index: 856

Text #16

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.5858. Min Act: -0.1700

Data Index: 4644346 (Open Web Text)

Max Activating Token Index: 798

Text #17

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.5933. Min Act: -0.1700

Data Index: 5855575 (Open Web Text)

Max Activating Token Index: 162

Text #18

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.6250. Min Act: -0.1700

Data Index: 400969 (Open Web Text)

Max Activating Token Index: 521

Text #19

Max Range: 3.9288. Min Range: -3.9288

Max Act: 3.5874. Min Act: -0.1700

Data Index: 4656643 (Open Web Text)

Max Activating Token Index: 1006