Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 843 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2585. Min Range: -3.2585

Max Act: 3.2585. Min Act: -0.1700

Data Index: 1405608 (Open Web Text)

Max Activating Token Index: 50

Text #1

Max Range: 3.2585. Min Range: -3.2585

Max Act: 3.2142. Min Act: -0.1700

Data Index: 6280047 (Open Web Text)

Max Activating Token Index: 448

Text #2

Max Range: 3.2585. Min Range: -3.2585

Max Act: 3.1624. Min Act: -0.1700

Data Index: 5835482 (Open Web Text)

Max Activating Token Index: 457

Text #3

Max Range: 3.2585. Min Range: -3.2585

Max Act: 3.1225. Min Act: -0.1700

Data Index: 3485177 (Open Web Text)

Max Activating Token Index: 718

Text #4

Max Range: 3.2585. Min Range: -3.2585

Max Act: 3.1072. Min Act: -0.1700

Data Index: 1366273 (Open Web Text)

Max Activating Token Index: 420

Text #5

Max Range: 3.2585. Min Range: -3.2585

Max Act: 3.0671. Min Act: -0.1700

Data Index: 6448263 (Open Web Text)

Max Activating Token Index: 674

Text #6

Max Range: 3.2585. Min Range: -3.2585

Max Act: 3.0781. Min Act: -0.1700

Data Index: 3965818 (Open Web Text)

Max Activating Token Index: 966

Text #7

Max Range: 3.2585. Min Range: -3.2585

Max Act: 3.0663. Min Act: -0.1700

Data Index: 1773583 (Open Web Text)

Max Activating Token Index: 729

Text #8

Max Range: 3.2585. Min Range: -3.2585

Max Act: 3.0602. Min Act: -0.1700

Data Index: 5263535 (Open Web Text)

Max Activating Token Index: 351

Text #9

Max Range: 3.2585. Min Range: -3.2585

Max Act: 3.0465. Min Act: -0.1700

Data Index: 2545921 (Open Web Text)

Max Activating Token Index: 452

Text #10

Max Range: 3.2585. Min Range: -3.2585

Max Act: 3.0188. Min Act: -0.1700

Data Index: 1707768 (Open Web Text)

Max Activating Token Index: 797

Text #11

Max Range: 3.2585. Min Range: -3.2585

Max Act: 3.0223. Min Act: -0.1700

Data Index: 3391178 (Open Web Text)

Max Activating Token Index: 259

Text #12

Max Range: 3.2585. Min Range: -3.2585

Max Act: 3.0289. Min Act: -0.1700

Data Index: 1376092 (Open Web Text)

Max Activating Token Index: 560

Text #13

Max Range: 3.2585. Min Range: -3.2585

Max Act: 2.9900. Min Act: -0.1700

Data Index: 2465904 (Open Web Text)

Max Activating Token Index: 930

Text #14

Max Range: 3.2585. Min Range: -3.2585

Max Act: 2.9789. Min Act: -0.1700

Data Index: 8512754 (Open Web Text)

Max Activating Token Index: 874

Text #15

Max Range: 3.2585. Min Range: -3.2585

Max Act: 2.9900. Min Act: -0.1700

Data Index: 1664173 (Open Web Text)

Max Activating Token Index: 386

Text #16

Max Range: 3.2585. Min Range: -3.2585

Max Act: 2.9889. Min Act: -0.1700

Data Index: 8587510 (Open Web Text)

Max Activating Token Index: 969

Text #17

Max Range: 3.2585. Min Range: -3.2585

Max Act: 2.9904. Min Act: -0.1700

Data Index: 7100921 (Open Web Text)

Max Activating Token Index: 363

Text #18

Max Range: 3.2585. Min Range: -3.2585

Max Act: 2.9884. Min Act: -0.1700

Data Index: 5638013 (Open Web Text)

Max Activating Token Index: 383

Text #19

Max Range: 3.2585. Min Range: -3.2585

Max Act: 2.9669. Min Act: -0.1700

Data Index: 5406013 (Open Web Text)

Max Activating Token Index: 715