Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 640 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.3169. Min Act: -0.1700

Data Index: 8166107 (Open Web Text)

Max Activating Token Index: 713

Text #1

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.2918. Min Act: -0.1700

Data Index: 6425426 (Open Web Text)

Max Activating Token Index: 50

Text #2

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.2440. Min Act: -0.1700

Data Index: 8592489 (Open Web Text)

Max Activating Token Index: 378

Text #3

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.2204. Min Act: -0.1700

Data Index: 566461 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.2349. Min Act: -0.1700

Data Index: 5133864 (Open Web Text)

Max Activating Token Index: 767

Text #5

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.1982. Min Act: -0.1700

Data Index: 6390013 (Open Web Text)

Max Activating Token Index: 811

Text #6

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.2064. Min Act: -0.1700

Data Index: 4297645 (Open Web Text)

Max Activating Token Index: 47

Text #7

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.2156. Min Act: -0.1700

Data Index: 397133 (Open Web Text)

Max Activating Token Index: 21

Text #8

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.1574. Min Act: -0.1700

Data Index: 7793868 (Open Web Text)

Max Activating Token Index: 70

Text #9

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.1680. Min Act: -0.1700

Data Index: 3187744 (Open Web Text)

Max Activating Token Index: 168

Text #10

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.1599. Min Act: -0.1700

Data Index: 700158 (Open Web Text)

Max Activating Token Index: 700

Text #11

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.1825. Min Act: -0.1700

Data Index: 40535 (Open Web Text)

Max Activating Token Index: 52

Text #12

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.1192. Min Act: -0.1700

Data Index: 2069222 (Open Web Text)

Max Activating Token Index: 4

Text #13

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.1316. Min Act: -0.1700

Data Index: 4156008 (Open Web Text)

Max Activating Token Index: 98

Text #14

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.1236. Min Act: -0.1700

Data Index: 8467647 (Open Web Text)

Max Activating Token Index: 578

Text #15

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.1205. Min Act: -0.1700

Data Index: 5056986 (Open Web Text)

Max Activating Token Index: 10

Text #16

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.1078. Min Act: -0.1700

Data Index: 2381202 (Open Web Text)

Max Activating Token Index: 57

Text #17

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.0867. Min Act: -0.1700

Data Index: 5983079 (Open Web Text)

Max Activating Token Index: 709

Text #18

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.0473. Min Act: -0.1700

Data Index: 5800174 (Open Web Text)

Max Activating Token Index: 452

Text #19

Max Range: 4.3169. Min Range: -4.3169

Max Act: 4.0722. Min Act: -0.1700

Data Index: 5988008 (Open Web Text)

Max Activating Token Index: 840