Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3018 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.9344. Min Act: -0.1700

Data Index: 3456327 (Open Web Text)

Max Activating Token Index: 196

Text #1

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.8854. Min Act: -0.1700

Data Index: 1232386 (Open Web Text)

Max Activating Token Index: 23

Text #2

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.8775. Min Act: -0.1700

Data Index: 7309367 (Open Web Text)

Max Activating Token Index: 343

Text #3

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.8527. Min Act: -0.1700

Data Index: 7406895 (Open Web Text)

Max Activating Token Index: 153

Text #4

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.8252. Min Act: -0.1700

Data Index: 1624500 (Open Web Text)

Max Activating Token Index: 675

Text #5

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.7886. Min Act: -0.1700

Data Index: 3781203 (Open Web Text)

Max Activating Token Index: 1013

Text #6

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.7780. Min Act: -0.1700

Data Index: 5399185 (Open Web Text)

Max Activating Token Index: 412

Text #7

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.7817. Min Act: -0.1700

Data Index: 3419384 (Open Web Text)

Max Activating Token Index: 169

Text #8

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.7826. Min Act: -0.1700

Data Index: 733626 (Open Web Text)

Max Activating Token Index: 908

Text #9

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.7440. Min Act: -0.1700

Data Index: 1951831 (Open Web Text)

Max Activating Token Index: 660

Text #10

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.7615. Min Act: -0.1700

Data Index: 3087799 (Open Web Text)

Max Activating Token Index: 330

Text #11

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.7398. Min Act: -0.1700

Data Index: 4703174 (Open Web Text)

Max Activating Token Index: 25

Text #12

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.7564. Min Act: -0.1700

Data Index: 2574006 (Open Web Text)

Max Activating Token Index: 478

Text #13

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.7482. Min Act: -0.1700

Data Index: 3916061 (Open Web Text)

Max Activating Token Index: 9

Text #14

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.7638. Min Act: -0.1700

Data Index: 8796246 (Open Web Text)

Max Activating Token Index: 14

Text #15

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.7439. Min Act: -0.1700

Data Index: 2508023 (Open Web Text)

Max Activating Token Index: 370

Text #16

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.7046. Min Act: -0.1700

Data Index: 6556282 (Open Web Text)

Max Activating Token Index: 386

Text #17

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.7274. Min Act: -0.1700

Data Index: 6419717 (Open Web Text)

Max Activating Token Index: 846

Text #18

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.7298. Min Act: -0.1700

Data Index: 335933 (Open Web Text)

Max Activating Token Index: 926

Text #19

Max Range: 5.9344. Min Range: -5.9344

Max Act: 5.7200. Min Act: -0.1700

Data Index: 3392167 (Open Web Text)

Max Activating Token Index: 627