Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1236 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.9241. Min Act: -0.1700

Data Index: 7100331 (Open Web Text)

Max Activating Token Index: 100

Text #1

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.9034. Min Act: -0.1700

Data Index: 5965237 (Open Web Text)

Max Activating Token Index: 233

Text #2

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.8911. Min Act: -0.1700

Data Index: 1452961 (Open Web Text)

Max Activating Token Index: 841

Text #3

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.8638. Min Act: -0.1700

Data Index: 1452962 (Open Web Text)

Max Activating Token Index: 293

Text #4

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.8717. Min Act: -0.1700

Data Index: 982022 (Open Web Text)

Max Activating Token Index: 389

Text #5

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.7962. Min Act: -0.1700

Data Index: 4496157 (Open Web Text)

Max Activating Token Index: 212

Text #6

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.8177. Min Act: -0.1700

Data Index: 2930950 (Open Web Text)

Max Activating Token Index: 330

Text #7

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.7744. Min Act: -0.1700

Data Index: 2545973 (Open Web Text)

Max Activating Token Index: 700

Text #8

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.7485. Min Act: -0.1700

Data Index: 1953704 (Open Web Text)

Max Activating Token Index: 153

Text #9

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.7315. Min Act: -0.1700

Data Index: 1295813 (Open Web Text)

Max Activating Token Index: 390

Text #10

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.7295. Min Act: -0.1700

Data Index: 775442 (Open Web Text)

Max Activating Token Index: 233

Text #11

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.7250. Min Act: -0.1700

Data Index: 1909518 (Open Web Text)

Max Activating Token Index: 227

Text #12

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.7399. Min Act: -0.1700

Data Index: 3903167 (Open Web Text)

Max Activating Token Index: 405

Text #13

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.6716. Min Act: -0.1700

Data Index: 2869863 (Open Web Text)

Max Activating Token Index: 713

Text #14

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.6611. Min Act: -0.1700

Data Index: 322640 (Open Web Text)

Max Activating Token Index: 673

Text #15

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.6783. Min Act: -0.1700

Data Index: 5036713 (Open Web Text)

Max Activating Token Index: 614

Text #16

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.6434. Min Act: -0.1700

Data Index: 3898325 (Open Web Text)

Max Activating Token Index: 341

Text #17

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.6435. Min Act: -0.1700

Data Index: 3201249 (Open Web Text)

Max Activating Token Index: 557

Text #18

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.6342. Min Act: -0.1700

Data Index: 2362440 (Open Web Text)

Max Activating Token Index: 524

Text #19

Max Range: 3.9241. Min Range: -3.9241

Max Act: 3.6036. Min Act: -0.1700

Data Index: 5093458 (Open Web Text)

Max Activating Token Index: 281