Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 607 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.3250. Min Act: -0.1700

Data Index: 506322 (Open Web Text)

Max Activating Token Index: 691

Text #1

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2872. Min Act: -0.1700

Data Index: 506330 (Open Web Text)

Max Activating Token Index: 663

Text #2

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2849. Min Act: -0.1700

Data Index: 506323 (Open Web Text)

Max Activating Token Index: 634

Text #3

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2636. Min Act: -0.1700

Data Index: 506321 (Open Web Text)

Max Activating Token Index: 748

Text #4

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2694. Min Act: -0.1700

Data Index: 506327 (Open Web Text)

Max Activating Token Index: 1019

Text #5

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2468. Min Act: -0.1700

Data Index: 506342 (Open Web Text)

Max Activating Token Index: 679

Text #6

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2420. Min Act: -0.1700

Data Index: 506331 (Open Web Text)

Max Activating Token Index: 610

Text #7

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2439. Min Act: -0.1700

Data Index: 506316 (Open Web Text)

Max Activating Token Index: 647

Text #8

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2472. Min Act: -0.1700

Data Index: 6525103 (Open Web Text)

Max Activating Token Index: 609

Text #9

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2429. Min Act: -0.1700

Data Index: 506325 (Open Web Text)

Max Activating Token Index: 673

Text #10

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2317. Min Act: -0.1700

Data Index: 506329 (Open Web Text)

Max Activating Token Index: 720

Text #11

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2288. Min Act: -0.1700

Data Index: 506328 (Open Web Text)

Max Activating Token Index: 628

Text #12

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2452. Min Act: -0.1700

Data Index: 506336 (Open Web Text)

Max Activating Token Index: 1021

Text #13

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2314. Min Act: -0.1700

Data Index: 506326 (Open Web Text)

Max Activating Token Index: 616

Text #14

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2244. Min Act: -0.1700

Data Index: 506333 (Open Web Text)

Max Activating Token Index: 689

Text #15

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2173. Min Act: -0.1700

Data Index: 506332 (Open Web Text)

Max Activating Token Index: 392

Text #16

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2130. Min Act: -0.1700

Data Index: 506335 (Open Web Text)

Max Activating Token Index: 756

Text #17

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2123. Min Act: -0.1700

Data Index: 506315 (Open Web Text)

Max Activating Token Index: 704

Text #18

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2042. Min Act: -0.1700

Data Index: 506339 (Open Web Text)

Max Activating Token Index: 689

Text #19

Max Range: 3.3250. Min Range: -3.3250

Max Act: 3.2063. Min Act: -0.1700

Data Index: 506320 (Open Web Text)

Max Activating Token Index: 636