Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1635 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1409. Min Range: -4.1409

Max Act: 4.1409. Min Act: -0.1700

Data Index: 2585460 (Open Web Text)

Max Activating Token Index: 904

Text #1

Max Range: 4.1409. Min Range: -4.1409

Max Act: 4.0531. Min Act: -0.1700

Data Index: 4048137 (Open Web Text)

Max Activating Token Index: 75

Text #2

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.9270. Min Act: -0.1700

Data Index: 2099399 (Open Web Text)

Max Activating Token Index: 448

Text #3

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.8648. Min Act: -0.1700

Data Index: 1170591 (Open Web Text)

Max Activating Token Index: 703

Text #4

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.7753. Min Act: -0.1700

Data Index: 3749819 (Open Web Text)

Max Activating Token Index: 564

Text #5

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.7343. Min Act: -0.1700

Data Index: 2424261 (Open Web Text)

Max Activating Token Index: 457

Text #6

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.7536. Min Act: -0.1700

Data Index: 5330812 (Open Web Text)

Max Activating Token Index: 280

Text #7

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.7536. Min Act: -0.1700

Data Index: 7869768 (Open Web Text)

Max Activating Token Index: 793

Text #8

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.7166. Min Act: -0.1700

Data Index: 4091916 (Open Web Text)

Max Activating Token Index: 811

Text #9

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.7843. Min Act: -0.1700

Data Index: 1306131 (Open Web Text)

Max Activating Token Index: 936

Text #10

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.7034. Min Act: -0.1700

Data Index: 6108050 (Open Web Text)

Max Activating Token Index: 652

Text #11

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.6740. Min Act: -0.1700

Data Index: 4397670 (Open Web Text)

Max Activating Token Index: 983

Text #12

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.7350. Min Act: -0.1700

Data Index: 8297236 (Open Web Text)

Max Activating Token Index: 192

Text #13

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.7223. Min Act: -0.1700

Data Index: 8087169 (Open Web Text)

Max Activating Token Index: 599

Text #14

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.6684. Min Act: -0.1700

Data Index: 261808 (Open Web Text)

Max Activating Token Index: 899

Text #15

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.6852. Min Act: -0.1700

Data Index: 1515789 (Open Web Text)

Max Activating Token Index: 968

Text #16

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.6902. Min Act: -0.1700

Data Index: 2446203 (Open Web Text)

Max Activating Token Index: 851

Text #17

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.6605. Min Act: -0.1700

Data Index: 4310837 (Open Web Text)

Max Activating Token Index: 97

Text #18

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.6703. Min Act: -0.1700

Data Index: 7685808 (Open Web Text)

Max Activating Token Index: 480

Text #19

Max Range: 4.1409. Min Range: -4.1409

Max Act: 3.6990. Min Act: -0.1700

Data Index: 3626093 (Open Web Text)

Max Activating Token Index: 21