Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 760 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.0989. Min Range: -2.0989

Max Act: 2.0989. Min Act: -0.1700

Data Index: 2614461 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.9935. Min Act: -0.1700

Data Index: 7149823 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.9600. Min Act: -0.1700

Data Index: 1993928 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.9345. Min Act: -0.1700

Data Index: 5684888 (Open Web Text)

Max Activating Token Index: 1023

Text #4

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.8709. Min Act: -0.1700

Data Index: 433536 (Open Web Text)

Max Activating Token Index: 760

Text #5

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.8789. Min Act: -0.1700

Data Index: 2657540 (Open Web Text)

Max Activating Token Index: 1023

Text #6

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.8681. Min Act: -0.1700

Data Index: 8177912 (Open Web Text)

Max Activating Token Index: 1023

Text #7

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.8663. Min Act: -0.1700

Data Index: 3982895 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.8462. Min Act: -0.1700

Data Index: 3270529 (Open Web Text)

Max Activating Token Index: 403

Text #9

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.8357. Min Act: -0.1700

Data Index: 5992868 (Open Web Text)

Max Activating Token Index: 1023

Text #10

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.7983. Min Act: -0.1700

Data Index: 4260519 (Open Web Text)

Max Activating Token Index: 659

Text #11

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.8183. Min Act: -0.1700

Data Index: 3637246 (Open Web Text)

Max Activating Token Index: 973

Text #12

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.8067. Min Act: -0.1700

Data Index: 8427277 (Open Web Text)

Max Activating Token Index: 1023

Text #13

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.7896. Min Act: -0.1700

Data Index: 4939116 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.8036. Min Act: -0.1700

Data Index: 4446301 (Open Web Text)

Max Activating Token Index: 446

Text #15

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.7644. Min Act: -0.1700

Data Index: 5626958 (Open Web Text)

Max Activating Token Index: 1023

Text #16

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.7762. Min Act: -0.1700

Data Index: 3731200 (Open Web Text)

Max Activating Token Index: 671

Text #17

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.7644. Min Act: -0.1700

Data Index: 1721982 (Open Web Text)

Max Activating Token Index: 1023

Text #18

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.7891. Min Act: -0.1700

Data Index: 6414124 (Open Web Text)

Max Activating Token Index: 1023

Text #19

Max Range: 2.0989. Min Range: -2.0989

Max Act: 1.7683. Min Act: -0.1700

Data Index: 6061822 (Open Web Text)

Max Activating Token Index: 1023