Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1966 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.2060. Min Act: -0.1700

Data Index: 6043315 (Open Web Text)

Max Activating Token Index: 310

Text #1

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.2126. Min Act: -0.1700

Data Index: 2663338 (Open Web Text)

Max Activating Token Index: 825

Text #2

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.1880. Min Act: -0.1700

Data Index: 4425974 (Open Web Text)

Max Activating Token Index: 227

Text #3

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.1570. Min Act: -0.1700

Data Index: 2918173 (Open Web Text)

Max Activating Token Index: 43

Text #4

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.1460. Min Act: -0.1700

Data Index: 5697722 (Open Web Text)

Max Activating Token Index: 502

Text #5

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.1141. Min Act: -0.1700

Data Index: 5151637 (Open Web Text)

Max Activating Token Index: 62

Text #6

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.0900. Min Act: -0.1700

Data Index: 2008131 (Open Web Text)

Max Activating Token Index: 55

Text #7

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.1293. Min Act: -0.1700

Data Index: 4747311 (Open Web Text)

Max Activating Token Index: 742

Text #8

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.0696. Min Act: -0.1700

Data Index: 6820041 (Open Web Text)

Max Activating Token Index: 354

Text #9

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.0725. Min Act: -0.1700

Data Index: 6897228 (Open Web Text)

Max Activating Token Index: 783

Text #10

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.0204. Min Act: -0.1700

Data Index: 2916600 (Open Web Text)

Max Activating Token Index: 11

Text #11

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.0389. Min Act: -0.1700

Data Index: 8752511 (Open Web Text)

Max Activating Token Index: 164

Text #12

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.0472. Min Act: -0.1700

Data Index: 4090169 (Open Web Text)

Max Activating Token Index: 624

Text #13

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.0146. Min Act: -0.1700

Data Index: 1152348 (Open Web Text)

Max Activating Token Index: 45

Text #14

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.0674. Min Act: -0.1700

Data Index: 5513822 (Open Web Text)

Max Activating Token Index: 488

Text #15

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.0283. Min Act: -0.1700

Data Index: 447307 (Open Web Text)

Max Activating Token Index: 40

Text #16

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.0042. Min Act: -0.1700

Data Index: 562058 (Open Web Text)

Max Activating Token Index: 167

Text #17

Max Range: 4.2126. Min Range: -4.2126

Max Act: 3.9980. Min Act: -0.1700

Data Index: 5161056 (Open Web Text)

Max Activating Token Index: 659

Text #18

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.0233. Min Act: -0.1700

Data Index: 3271614 (Open Web Text)

Max Activating Token Index: 117

Text #19

Max Range: 4.2126. Min Range: -4.2126

Max Act: 4.0031. Min Act: -0.1700

Data Index: 2098634 (Open Web Text)

Max Activating Token Index: 342