Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3006 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.3329. Min Act: -0.1700

Data Index: 1450010 (Open Web Text)

Max Activating Token Index: 543

Text #1

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2865. Min Act: -0.1700

Data Index: 2569303 (Open Web Text)

Max Activating Token Index: 423

Text #2

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2721. Min Act: -0.1700

Data Index: 2354090 (Open Web Text)

Max Activating Token Index: 108

Text #3

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2661. Min Act: -0.1700

Data Index: 7958621 (Open Web Text)

Max Activating Token Index: 911

Text #4

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2508. Min Act: -0.1700

Data Index: 821912 (Open Web Text)

Max Activating Token Index: 530

Text #5

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2499. Min Act: -0.1700

Data Index: 1246185 (Open Web Text)

Max Activating Token Index: 757

Text #6

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2527. Min Act: -0.1700

Data Index: 4124138 (Open Web Text)

Max Activating Token Index: 506

Text #7

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2353. Min Act: -0.1700

Data Index: 1850141 (Open Web Text)

Max Activating Token Index: 240

Text #8

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2553. Min Act: -0.1700

Data Index: 3638543 (Open Web Text)

Max Activating Token Index: 578

Text #9

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2515. Min Act: -0.1700

Data Index: 4910462 (Open Web Text)

Max Activating Token Index: 752

Text #10

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2533. Min Act: -0.1700

Data Index: 4807711 (Open Web Text)

Max Activating Token Index: 255

Text #11

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2268. Min Act: -0.1700

Data Index: 6238355 (Open Web Text)

Max Activating Token Index: 547

Text #12

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2419. Min Act: -0.1700

Data Index: 1317632 (Open Web Text)

Max Activating Token Index: 574

Text #13

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2334. Min Act: -0.1700

Data Index: 207295 (Open Web Text)

Max Activating Token Index: 273

Text #14

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2119. Min Act: -0.1700

Data Index: 933164 (Open Web Text)

Max Activating Token Index: 491

Text #15

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2110. Min Act: -0.1700

Data Index: 6715184 (Open Web Text)

Max Activating Token Index: 50

Text #16

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2099. Min Act: -0.1700

Data Index: 3656688 (Open Web Text)

Max Activating Token Index: 511

Text #17

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2188. Min Act: -0.1700

Data Index: 4247268 (Open Web Text)

Max Activating Token Index: 714

Text #18

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2087. Min Act: -0.1700

Data Index: 4641045 (Open Web Text)

Max Activating Token Index: 340

Text #19

Max Range: 2.3329. Min Range: -2.3329

Max Act: 2.2193. Min Act: -0.1700

Data Index: 8068787 (Open Web Text)

Max Activating Token Index: 405