Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2849 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1879. Min Range: -4.1879

Max Act: 4.1879. Min Act: -0.1700

Data Index: 469443 (Open Web Text)

Max Activating Token Index: 459

Text #1

Max Range: 4.1879. Min Range: -4.1879

Max Act: 4.0678. Min Act: -0.1700

Data Index: 7881709 (Open Web Text)

Max Activating Token Index: 990

Text #2

Max Range: 4.1879. Min Range: -4.1879

Max Act: 4.0617. Min Act: -0.1700

Data Index: 5099358 (Open Web Text)

Max Activating Token Index: 570

Text #3

Max Range: 4.1879. Min Range: -4.1879

Max Act: 3.9914. Min Act: -0.1700

Data Index: 8827989 (Open Web Text)

Max Activating Token Index: 167

Text #4

Max Range: 4.1879. Min Range: -4.1879

Max Act: 4.0498. Min Act: -0.1700

Data Index: 2718092 (Open Web Text)

Max Activating Token Index: 79

Text #5

Max Range: 4.1879. Min Range: -4.1879

Max Act: 4.0144. Min Act: -0.1700

Data Index: 2365448 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 4.1879. Min Range: -4.1879

Max Act: 4.0327. Min Act: -0.1700

Data Index: 3572147 (Open Web Text)

Max Activating Token Index: 781

Text #7

Max Range: 4.1879. Min Range: -4.1879

Max Act: 4.0144. Min Act: -0.1700

Data Index: 1828126 (Open Web Text)

Max Activating Token Index: 3

Text #8

Max Range: 4.1879. Min Range: -4.1879

Max Act: 4.0357. Min Act: -0.1700

Data Index: 4580374 (Open Web Text)

Max Activating Token Index: 76

Text #9

Max Range: 4.1879. Min Range: -4.1879

Max Act: 3.9976. Min Act: -0.1700

Data Index: 984258 (Open Web Text)

Max Activating Token Index: 896

Text #10

Max Range: 4.1879. Min Range: -4.1879

Max Act: 4.0001. Min Act: -0.1700

Data Index: 5441655 (Open Web Text)

Max Activating Token Index: 173

Text #11

Max Range: 4.1879. Min Range: -4.1879

Max Act: 3.9972. Min Act: -0.1700

Data Index: 5762135 (Open Web Text)

Max Activating Token Index: 975

Text #12

Max Range: 4.1879. Min Range: -4.1879

Max Act: 3.9925. Min Act: -0.1700

Data Index: 5074835 (Open Web Text)

Max Activating Token Index: 90

Text #13

Max Range: 4.1879. Min Range: -4.1879

Max Act: 4.0304. Min Act: -0.1700

Data Index: 2193503 (Open Web Text)

Max Activating Token Index: 223

Text #14

Max Range: 4.1879. Min Range: -4.1879

Max Act: 3.9906. Min Act: -0.1700

Data Index: 6781361 (Open Web Text)

Max Activating Token Index: 254

Text #15

Max Range: 4.1879. Min Range: -4.1879

Max Act: 3.9965. Min Act: -0.1700

Data Index: 6806327 (Open Web Text)

Max Activating Token Index: 469

Text #16

Max Range: 4.1879. Min Range: -4.1879

Max Act: 3.9796. Min Act: -0.1700

Data Index: 3336787 (Open Web Text)

Max Activating Token Index: 257

Text #17

Max Range: 4.1879. Min Range: -4.1879

Max Act: 3.9693. Min Act: -0.1700

Data Index: 5026707 (Open Web Text)

Max Activating Token Index: 177

Text #18

Max Range: 4.1879. Min Range: -4.1879

Max Act: 3.9572. Min Act: -0.1700

Data Index: 5908413 (Open Web Text)

Max Activating Token Index: 402

Text #19

Max Range: 4.1879. Min Range: -4.1879

Max Act: 3.9911. Min Act: -0.1700

Data Index: 2660391 (Open Web Text)

Max Activating Token Index: 641