Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 496 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0091. Min Range: -3.0091

Max Act: 3.0091. Min Act: -0.1700

Data Index: 8729728 (Open Web Text)

Max Activating Token Index: 8

Text #1

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.9941. Min Act: -0.1700

Data Index: 3485130 (Open Web Text)

Max Activating Token Index: 711

Text #2

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.9669. Min Act: -0.1700

Data Index: 3644809 (Open Web Text)

Max Activating Token Index: 23

Text #3

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.9446. Min Act: -0.1700

Data Index: 2332191 (Open Web Text)

Max Activating Token Index: 112

Text #4

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.9445. Min Act: -0.1700

Data Index: 7759864 (Open Web Text)

Max Activating Token Index: 7

Text #5

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.9298. Min Act: -0.1700

Data Index: 7648139 (Open Web Text)

Max Activating Token Index: 703

Text #6

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.9209. Min Act: -0.1700

Data Index: 7108023 (Open Web Text)

Max Activating Token Index: 16

Text #7

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.9296. Min Act: -0.1700

Data Index: 2508013 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.9208. Min Act: -0.1700

Data Index: 4915901 (Open Web Text)

Max Activating Token Index: 9

Text #9

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.9251. Min Act: -0.1700

Data Index: 2903803 (Open Web Text)

Max Activating Token Index: 689

Text #10

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.9234. Min Act: -0.1700

Data Index: 733178 (Open Web Text)

Max Activating Token Index: 675

Text #11

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.9187. Min Act: -0.1700

Data Index: 6634234 (Open Web Text)

Max Activating Token Index: 18

Text #12

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.9222. Min Act: -0.1700

Data Index: 5571081 (Open Web Text)

Max Activating Token Index: 24

Text #13

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.9038. Min Act: -0.1700

Data Index: 4219781 (Open Web Text)

Max Activating Token Index: 8

Text #14

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.8909. Min Act: -0.1700

Data Index: 2126813 (Open Web Text)

Max Activating Token Index: 761

Text #15

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.9022. Min Act: -0.1700

Data Index: 6970411 (Open Web Text)

Max Activating Token Index: 101

Text #16

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.8906. Min Act: -0.1700

Data Index: 7981264 (Open Web Text)

Max Activating Token Index: 180

Text #17

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.8930. Min Act: -0.1700

Data Index: 1956874 (Open Web Text)

Max Activating Token Index: 360

Text #18

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.8940. Min Act: -0.1700

Data Index: 5194579 (Open Web Text)

Max Activating Token Index: 9

Text #19

Max Range: 3.0091. Min Range: -3.0091

Max Act: 2.8810. Min Act: -0.1700

Data Index: 1758971 (Open Web Text)

Max Activating Token Index: 243