Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 120 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1297. Min Range: -3.1297

Max Act: 3.1297. Min Act: -0.1700

Data Index: 2386687 (Open Web Text)

Max Activating Token Index: 678

Text #1

Max Range: 3.1297. Min Range: -3.1297

Max Act: 3.0982. Min Act: -0.1700

Data Index: 5143231 (Open Web Text)

Max Activating Token Index: 2

Text #2

Max Range: 3.1297. Min Range: -3.1297

Max Act: 3.0549. Min Act: -0.1700

Data Index: 7158493 (Open Web Text)

Max Activating Token Index: 688

Text #3

Max Range: 3.1297. Min Range: -3.1297

Max Act: 3.0381. Min Act: -0.1700

Data Index: 3334762 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 3.1297. Min Range: -3.1297

Max Act: 3.0359. Min Act: -0.1700

Data Index: 967772 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 3.1297. Min Range: -3.1297

Max Act: 3.0274. Min Act: -0.1700

Data Index: 650776 (Open Web Text)

Max Activating Token Index: 356

Text #6

Max Range: 3.1297. Min Range: -3.1297

Max Act: 3.0035. Min Act: -0.1700

Data Index: 5390790 (Open Web Text)

Max Activating Token Index: 220

Text #7

Max Range: 3.1297. Min Range: -3.1297

Max Act: 3.0030. Min Act: -0.1700

Data Index: 6155025 (Open Web Text)

Max Activating Token Index: 378

Text #8

Max Range: 3.1297. Min Range: -3.1297

Max Act: 2.9883. Min Act: -0.1700

Data Index: 3539216 (Open Web Text)

Max Activating Token Index: 513

Text #9

Max Range: 3.1297. Min Range: -3.1297

Max Act: 2.9883. Min Act: -0.1700

Data Index: 5380729 (Open Web Text)

Max Activating Token Index: 364

Text #10

Max Range: 3.1297. Min Range: -3.1297

Max Act: 2.9680. Min Act: -0.1700

Data Index: 103097 (Open Web Text)

Max Activating Token Index: 713

Text #11

Max Range: 3.1297. Min Range: -3.1297

Max Act: 2.9735. Min Act: -0.1700

Data Index: 3026025 (Open Web Text)

Max Activating Token Index: 801

Text #12

Max Range: 3.1297. Min Range: -3.1297

Max Act: 2.9681. Min Act: -0.1700

Data Index: 7201668 (Open Web Text)

Max Activating Token Index: 659

Text #13

Max Range: 3.1297. Min Range: -3.1297

Max Act: 2.9569. Min Act: -0.1700

Data Index: 8384152 (Open Web Text)

Max Activating Token Index: 704

Text #14

Max Range: 3.1297. Min Range: -3.1297

Max Act: 2.9578. Min Act: -0.1700

Data Index: 8665660 (Open Web Text)

Max Activating Token Index: 641

Text #15

Max Range: 3.1297. Min Range: -3.1297

Max Act: 2.9674. Min Act: -0.1700

Data Index: 7104552 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 3.1297. Min Range: -3.1297

Max Act: 2.9696. Min Act: -0.1700

Data Index: 995453 (Open Web Text)

Max Activating Token Index: 917

Text #17

Max Range: 3.1297. Min Range: -3.1297

Max Act: 2.9653. Min Act: -0.1700

Data Index: 1634139 (Open Web Text)

Max Activating Token Index: 535

Text #18

Max Range: 3.1297. Min Range: -3.1297

Max Act: 2.9512. Min Act: -0.1700

Data Index: 821680 (Open Web Text)

Max Activating Token Index: 657

Text #19

Max Range: 3.1297. Min Range: -3.1297

Max Act: 2.9384. Min Act: -0.1700

Data Index: 5149491 (Open Web Text)

Max Activating Token Index: 455