Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1845 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.5441. Min Act: -0.1700

Data Index: 6906044 (Open Web Text)

Max Activating Token Index: 64

Text #1

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.3888. Min Act: -0.1700

Data Index: 7485482 (Open Web Text)

Max Activating Token Index: 739

Text #2

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.3461. Min Act: -0.1700

Data Index: 1286308 (Open Web Text)

Max Activating Token Index: 983

Text #3

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.2911. Min Act: -0.1700

Data Index: 4159628 (Open Web Text)

Max Activating Token Index: 192

Text #4

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.2547. Min Act: -0.1700

Data Index: 3619993 (Open Web Text)

Max Activating Token Index: 312

Text #5

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.2287. Min Act: -0.1700

Data Index: 5195619 (Open Web Text)

Max Activating Token Index: 283

Text #6

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.2295. Min Act: -0.1700

Data Index: 3803614 (Open Web Text)

Max Activating Token Index: 397

Text #7

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.2195. Min Act: -0.1700

Data Index: 6327200 (Open Web Text)

Max Activating Token Index: 372

Text #8

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.2324. Min Act: -0.1700

Data Index: 4208078 (Open Web Text)

Max Activating Token Index: 855

Text #9

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.2027. Min Act: -0.1700

Data Index: 8705741 (Open Web Text)

Max Activating Token Index: 576

Text #10

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.2001. Min Act: -0.1700

Data Index: 3325279 (Open Web Text)

Max Activating Token Index: 192

Text #11

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.1968. Min Act: -0.1700

Data Index: 6631545 (Open Web Text)

Max Activating Token Index: 287

Text #12

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.1895. Min Act: -0.1700

Data Index: 6603654 (Open Web Text)

Max Activating Token Index: 168

Text #13

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.1884. Min Act: -0.1700

Data Index: 7380172 (Open Web Text)

Max Activating Token Index: 278

Text #14

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.2071. Min Act: -0.1700

Data Index: 504722 (Open Web Text)

Max Activating Token Index: 153

Text #15

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.1931. Min Act: -0.1700

Data Index: 5958944 (Open Web Text)

Max Activating Token Index: 722

Text #16

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.1495. Min Act: -0.1700

Data Index: 4270083 (Open Web Text)

Max Activating Token Index: 908

Text #17

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.1702. Min Act: -0.1700

Data Index: 4842822 (Open Web Text)

Max Activating Token Index: 343

Text #18

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.1907. Min Act: -0.1700

Data Index: 6057065 (Open Web Text)

Max Activating Token Index: 549

Text #19

Max Range: 3.5441. Min Range: -3.5441

Max Act: 3.1562. Min Act: -0.1700

Data Index: 5954350 (Open Web Text)

Max Activating Token Index: 51