Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 45 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.4466. Min Act: -0.1700

Data Index: 990271 (Open Web Text)

Max Activating Token Index: 365

Text #1

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.3460. Min Act: -0.1700

Data Index: 299531 (Open Web Text)

Max Activating Token Index: 451

Text #2

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.2779. Min Act: -0.1700

Data Index: 290872 (Open Web Text)

Max Activating Token Index: 655

Text #3

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.2965. Min Act: -0.1700

Data Index: 8050218 (Open Web Text)

Max Activating Token Index: 980

Text #4

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.2517. Min Act: -0.1700

Data Index: 3408793 (Open Web Text)

Max Activating Token Index: 855

Text #5

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.2422. Min Act: -0.1700

Data Index: 3227225 (Open Web Text)

Max Activating Token Index: 605

Text #6

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.2487. Min Act: -0.1700

Data Index: 8604338 (Open Web Text)

Max Activating Token Index: 169

Text #7

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.2355. Min Act: -0.1700

Data Index: 5174319 (Open Web Text)

Max Activating Token Index: 574

Text #8

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.2199. Min Act: -0.1700

Data Index: 6419734 (Open Web Text)

Max Activating Token Index: 288

Text #9

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.2081. Min Act: -0.1700

Data Index: 8087141 (Open Web Text)

Max Activating Token Index: 25

Text #10

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.2189. Min Act: -0.1700

Data Index: 6959084 (Open Web Text)

Max Activating Token Index: 643

Text #11

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.2098. Min Act: -0.1700

Data Index: 2191782 (Open Web Text)

Max Activating Token Index: 240

Text #12

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.1909. Min Act: -0.1700

Data Index: 5795322 (Open Web Text)

Max Activating Token Index: 507

Text #13

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.1967. Min Act: -0.1700

Data Index: 134371 (Open Web Text)

Max Activating Token Index: 498

Text #14

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.1920. Min Act: -0.1700

Data Index: 4962974 (Open Web Text)

Max Activating Token Index: 240

Text #15

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.1626. Min Act: -0.1700

Data Index: 2453473 (Open Web Text)

Max Activating Token Index: 914

Text #16

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.1748. Min Act: -0.1700

Data Index: 3299933 (Open Web Text)

Max Activating Token Index: 546

Text #17

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.1636. Min Act: -0.1700

Data Index: 797993 (Open Web Text)

Max Activating Token Index: 404

Text #18

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.1724. Min Act: -0.1700

Data Index: 8604337 (Open Web Text)

Max Activating Token Index: 416

Text #19

Max Range: 3.4466. Min Range: -3.4466

Max Act: 3.1440. Min Act: -0.1700

Data Index: 4355661 (Open Web Text)

Max Activating Token Index: 567