Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1110 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.5715. Min Act: -0.1700

Data Index: 1818641 (Open Web Text)

Max Activating Token Index: 660

Text #1

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.3580. Min Act: -0.1700

Data Index: 633026 (Open Web Text)

Max Activating Token Index: 670

Text #2

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.3509. Min Act: -0.1700

Data Index: 7031863 (Open Web Text)

Max Activating Token Index: 290

Text #3

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.3283. Min Act: -0.1700

Data Index: 4713594 (Open Web Text)

Max Activating Token Index: 461

Text #4

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.3245. Min Act: -0.1700

Data Index: 3039317 (Open Web Text)

Max Activating Token Index: 56

Text #5

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.3274. Min Act: -0.1700

Data Index: 3010635 (Open Web Text)

Max Activating Token Index: 571

Text #6

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.2911. Min Act: -0.1700

Data Index: 2790229 (Open Web Text)

Max Activating Token Index: 752

Text #7

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.3191. Min Act: -0.1700

Data Index: 8109587 (Open Web Text)

Max Activating Token Index: 778

Text #8

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.3243. Min Act: -0.1700

Data Index: 8770786 (Open Web Text)

Max Activating Token Index: 738

Text #9

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.3095. Min Act: -0.1700

Data Index: 2049852 (Open Web Text)

Max Activating Token Index: 820

Text #10

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.2886. Min Act: -0.1700

Data Index: 8444499 (Open Web Text)

Max Activating Token Index: 246

Text #11

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.2771. Min Act: -0.1700

Data Index: 5122568 (Open Web Text)

Max Activating Token Index: 120

Text #12

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.2533. Min Act: -0.1700

Data Index: 8451158 (Open Web Text)

Max Activating Token Index: 16

Text #13

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.2688. Min Act: -0.1700

Data Index: 5034439 (Open Web Text)

Max Activating Token Index: 717

Text #14

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.2365. Min Act: -0.1700

Data Index: 381677 (Open Web Text)

Max Activating Token Index: 451

Text #15

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.2695. Min Act: -0.1700

Data Index: 6320725 (Open Web Text)

Max Activating Token Index: 246

Text #16

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.2654. Min Act: -0.1700

Data Index: 8170043 (Open Web Text)

Max Activating Token Index: 194

Text #17

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.2567. Min Act: -0.1700

Data Index: 888694 (Open Web Text)

Max Activating Token Index: 326

Text #18

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.2497. Min Act: -0.1700

Data Index: 7862558 (Open Web Text)

Max Activating Token Index: 736

Text #19

Max Range: 3.5715. Min Range: -3.5715

Max Act: 3.2482. Min Act: -0.1700

Data Index: 4486342 (Open Web Text)

Max Activating Token Index: 115