Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2280 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1263. Min Range: -5.1263

Max Act: 5.1263. Min Act: -0.1700

Data Index: 3586074 (Open Web Text)

Max Activating Token Index: 585

Text #1

Max Range: 5.1263. Min Range: -5.1263

Max Act: 5.1193. Min Act: -0.1700

Data Index: 4191381 (Open Web Text)

Max Activating Token Index: 833

Text #2

Max Range: 5.1263. Min Range: -5.1263

Max Act: 5.0717. Min Act: -0.1700

Data Index: 7127127 (Open Web Text)

Max Activating Token Index: 935

Text #3

Max Range: 5.1263. Min Range: -5.1263

Max Act: 5.0405. Min Act: -0.1700

Data Index: 4226610 (Open Web Text)

Max Activating Token Index: 278

Text #4

Max Range: 5.1263. Min Range: -5.1263

Max Act: 5.0377. Min Act: -0.1700

Data Index: 3586075 (Open Web Text)

Max Activating Token Index: 1016

Text #5

Max Range: 5.1263. Min Range: -5.1263

Max Act: 4.9874. Min Act: -0.1700

Data Index: 8197262 (Open Web Text)

Max Activating Token Index: 400

Text #6

Max Range: 5.1263. Min Range: -5.1263

Max Act: 4.9348. Min Act: -0.1700

Data Index: 674178 (Open Web Text)

Max Activating Token Index: 517

Text #7

Max Range: 5.1263. Min Range: -5.1263

Max Act: 4.9081. Min Act: -0.1700

Data Index: 2064093 (Open Web Text)

Max Activating Token Index: 344

Text #8

Max Range: 5.1263. Min Range: -5.1263

Max Act: 4.9198. Min Act: -0.1700

Data Index: 6616411 (Open Web Text)

Max Activating Token Index: 834

Text #9

Max Range: 5.1263. Min Range: -5.1263

Max Act: 4.9350. Min Act: -0.1700

Data Index: 8561089 (Open Web Text)

Max Activating Token Index: 653

Text #10

Max Range: 5.1263. Min Range: -5.1263

Max Act: 4.9111. Min Act: -0.1700

Data Index: 5777781 (Open Web Text)

Max Activating Token Index: 1008

Text #11

Max Range: 5.1263. Min Range: -5.1263

Max Act: 4.9146. Min Act: -0.1700

Data Index: 4651862 (Open Web Text)

Max Activating Token Index: 193

Text #12

Max Range: 5.1263. Min Range: -5.1263

Max Act: 4.9080. Min Act: -0.1700

Data Index: 5542850 (Open Web Text)

Max Activating Token Index: 217

Text #13

Max Range: 5.1263. Min Range: -5.1263

Max Act: 4.8895. Min Act: -0.1700

Data Index: 2064091 (Open Web Text)

Max Activating Token Index: 1001

Text #14

Max Range: 5.1263. Min Range: -5.1263

Max Act: 4.9045. Min Act: -0.1700

Data Index: 6367709 (Open Web Text)

Max Activating Token Index: 878

Text #15

Max Range: 5.1263. Min Range: -5.1263

Max Act: 4.9119. Min Act: -0.1700

Data Index: 6329226 (Open Web Text)

Max Activating Token Index: 489

Text #16

Max Range: 5.1263. Min Range: -5.1263

Max Act: 4.8853. Min Act: -0.1700

Data Index: 4899767 (Open Web Text)

Max Activating Token Index: 24

Text #17

Max Range: 5.1263. Min Range: -5.1263

Max Act: 4.8814. Min Act: -0.1700

Data Index: 4938020 (Open Web Text)

Max Activating Token Index: 67

Text #18

Max Range: 5.1263. Min Range: -5.1263

Max Act: 4.8898. Min Act: -0.1700

Data Index: 6644775 (Open Web Text)

Max Activating Token Index: 458

Text #19

Max Range: 5.1263. Min Range: -5.1263

Max Act: 4.8673. Min Act: -0.1700

Data Index: 5033578 (Open Web Text)

Max Activating Token Index: 64