Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 352 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3914. Min Range: -4.3914

Max Act: 4.3914. Min Act: -0.1700

Data Index: 102826 (Open Web Text)

Max Activating Token Index: 682

Text #1

Max Range: 4.3914. Min Range: -4.3914

Max Act: 4.3779. Min Act: -0.1700

Data Index: 2326904 (Open Web Text)

Max Activating Token Index: 527

Text #2

Max Range: 4.3914. Min Range: -4.3914

Max Act: 4.2935. Min Act: -0.1700

Data Index: 1398324 (Open Web Text)

Max Activating Token Index: 564

Text #3

Max Range: 4.3914. Min Range: -4.3914

Max Act: 4.1566. Min Act: -0.1700

Data Index: 3398283 (Open Web Text)

Max Activating Token Index: 82

Text #4

Max Range: 4.3914. Min Range: -4.3914

Max Act: 4.1279. Min Act: -0.1700

Data Index: 7797418 (Open Web Text)

Max Activating Token Index: 637

Text #5

Max Range: 4.3914. Min Range: -4.3914

Max Act: 4.1481. Min Act: -0.1700

Data Index: 2439804 (Open Web Text)

Max Activating Token Index: 960

Text #6

Max Range: 4.3914. Min Range: -4.3914

Max Act: 4.0925. Min Act: -0.1700

Data Index: 4137562 (Open Web Text)

Max Activating Token Index: 95

Text #7

Max Range: 4.3914. Min Range: -4.3914

Max Act: 3.9956. Min Act: -0.1700

Data Index: 7453594 (Open Web Text)

Max Activating Token Index: 878

Text #8

Max Range: 4.3914. Min Range: -4.3914

Max Act: 3.9944. Min Act: -0.1700

Data Index: 1257256 (Open Web Text)

Max Activating Token Index: 675

Text #9

Max Range: 4.3914. Min Range: -4.3914

Max Act: 4.0183. Min Act: -0.1700

Data Index: 7289041 (Open Web Text)

Max Activating Token Index: 583

Text #10

Max Range: 4.3914. Min Range: -4.3914

Max Act: 4.0438. Min Act: -0.1700

Data Index: 2665831 (Open Web Text)

Max Activating Token Index: 767

Text #11

Max Range: 4.3914. Min Range: -4.3914

Max Act: 4.0029. Min Act: -0.1700

Data Index: 5976349 (Open Web Text)

Max Activating Token Index: 43

Text #12

Max Range: 4.3914. Min Range: -4.3914

Max Act: 3.9908. Min Act: -0.1700

Data Index: 5597555 (Open Web Text)

Max Activating Token Index: 577

Text #13

Max Range: 4.3914. Min Range: -4.3914

Max Act: 3.9621. Min Act: -0.1700

Data Index: 1991705 (Open Web Text)

Max Activating Token Index: 259

Text #14

Max Range: 4.3914. Min Range: -4.3914

Max Act: 4.0088. Min Act: -0.1700

Data Index: 2545354 (Open Web Text)

Max Activating Token Index: 965

Text #15

Max Range: 4.3914. Min Range: -4.3914

Max Act: 3.9914. Min Act: -0.1700

Data Index: 5509483 (Open Web Text)

Max Activating Token Index: 827

Text #16

Max Range: 4.3914. Min Range: -4.3914

Max Act: 3.9604. Min Act: -0.1700

Data Index: 253489 (Open Web Text)

Max Activating Token Index: 430

Text #17

Max Range: 4.3914. Min Range: -4.3914

Max Act: 3.9798. Min Act: -0.1700

Data Index: 5583797 (Open Web Text)

Max Activating Token Index: 150

Text #18

Max Range: 4.3914. Min Range: -4.3914

Max Act: 3.9560. Min Act: -0.1700

Data Index: 3740266 (Open Web Text)

Max Activating Token Index: 491

Text #19

Max Range: 4.3914. Min Range: -4.3914

Max Act: 3.9273. Min Act: -0.1700

Data Index: 5444281 (Open Web Text)

Max Activating Token Index: 283