Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 683 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.8800. Min Act: -0.1700

Data Index: 1051127 (Open Web Text)

Max Activating Token Index: 15

Text #1

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.8763. Min Act: -0.1700

Data Index: 7023128 (Open Web Text)

Max Activating Token Index: 33

Text #2

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.8189. Min Act: -0.1700

Data Index: 672223 (Open Web Text)

Max Activating Token Index: 14

Text #3

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.8285. Min Act: -0.1700

Data Index: 7914438 (Open Web Text)

Max Activating Token Index: 8

Text #4

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.8190. Min Act: -0.1700

Data Index: 4969683 (Open Web Text)

Max Activating Token Index: 2

Text #5

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.8175. Min Act: -0.1700

Data Index: 1391011 (Open Web Text)

Max Activating Token Index: 9

Text #6

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.8226. Min Act: -0.1700

Data Index: 5278823 (Open Web Text)

Max Activating Token Index: 7

Text #7

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.8034. Min Act: -0.1700

Data Index: 8574843 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.8080. Min Act: -0.1700

Data Index: 1303388 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.7967. Min Act: -0.1700

Data Index: 2662390 (Open Web Text)

Max Activating Token Index: 6

Text #10

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.8025. Min Act: -0.1700

Data Index: 1219423 (Open Web Text)

Max Activating Token Index: 23

Text #11

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.8064. Min Act: -0.1700

Data Index: 6543210 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.7888. Min Act: -0.1700

Data Index: 999218 (Open Web Text)

Max Activating Token Index: 15

Text #13

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.7863. Min Act: -0.1700

Data Index: 7796808 (Open Web Text)

Max Activating Token Index: 71

Text #14

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.7838. Min Act: -0.1700

Data Index: 5564440 (Open Web Text)

Max Activating Token Index: 704

Text #15

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.7820. Min Act: -0.1700

Data Index: 5236779 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.7751. Min Act: -0.1700

Data Index: 5075293 (Open Web Text)

Max Activating Token Index: 975

Text #17

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.7828. Min Act: -0.1700

Data Index: 5026243 (Open Web Text)

Max Activating Token Index: 103

Text #18

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.7752. Min Act: -0.1700

Data Index: 1072674 (Open Web Text)

Max Activating Token Index: 9

Text #19

Max Range: 2.8800. Min Range: -2.8800

Max Act: 2.7700. Min Act: -0.1700

Data Index: 8557350 (Open Web Text)

Max Activating Token Index: 4