Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1286 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2150. Min Range: -3.2150

Max Act: 3.2150. Min Act: -0.1700

Data Index: 8169425 (Open Web Text)

Max Activating Token Index: 362

Text #1

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.9745. Min Act: -0.1700

Data Index: 950878 (Open Web Text)

Max Activating Token Index: 378

Text #2

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.9173. Min Act: -0.1700

Data Index: 8475700 (Open Web Text)

Max Activating Token Index: 385

Text #3

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.9404. Min Act: -0.1700

Data Index: 5537750 (Open Web Text)

Max Activating Token Index: 441

Text #4

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.9216. Min Act: -0.1700

Data Index: 3786 (Open Web Text)

Max Activating Token Index: 401

Text #5

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.9139. Min Act: -0.1700

Data Index: 244367 (Open Web Text)

Max Activating Token Index: 329

Text #6

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.8771. Min Act: -0.1700

Data Index: 4242398 (Open Web Text)

Max Activating Token Index: 357

Text #7

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.8960. Min Act: -0.1700

Data Index: 8260676 (Open Web Text)

Max Activating Token Index: 369

Text #8

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.8819. Min Act: -0.1700

Data Index: 5165488 (Open Web Text)

Max Activating Token Index: 417

Text #9

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.8522. Min Act: -0.1700

Data Index: 1592587 (Open Web Text)

Max Activating Token Index: 383

Text #10

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.8350. Min Act: -0.1700

Data Index: 2716836 (Open Web Text)

Max Activating Token Index: 392

Text #11

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.8618. Min Act: -0.1700

Data Index: 5176009 (Open Web Text)

Max Activating Token Index: 392

Text #12

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.8436. Min Act: -0.1700

Data Index: 3049268 (Open Web Text)

Max Activating Token Index: 402

Text #13

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.8302. Min Act: -0.1700

Data Index: 6967991 (Open Web Text)

Max Activating Token Index: 444

Text #14

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.8607. Min Act: -0.1700

Data Index: 979076 (Open Web Text)

Max Activating Token Index: 427

Text #15

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.8140. Min Act: -0.1700

Data Index: 5552952 (Open Web Text)

Max Activating Token Index: 389

Text #16

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.8371. Min Act: -0.1700

Data Index: 255346 (Open Web Text)

Max Activating Token Index: 442

Text #17

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.8194. Min Act: -0.1700

Data Index: 6573819 (Open Web Text)

Max Activating Token Index: 451

Text #18

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.8497. Min Act: -0.1700

Data Index: 5742924 (Open Web Text)

Max Activating Token Index: 460

Text #19

Max Range: 3.2150. Min Range: -3.2150

Max Act: 2.7950. Min Act: -0.1700

Data Index: 7954072 (Open Web Text)

Max Activating Token Index: 341