Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2053 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.1026. Min Range: -6.1026

Max Act: 6.1026. Min Act: -0.1700

Data Index: 4654456 (Open Web Text)

Max Activating Token Index: 810

Text #1

Max Range: 6.1026. Min Range: -6.1026

Max Act: 6.0445. Min Act: -0.1700

Data Index: 1290842 (Open Web Text)

Max Activating Token Index: 1000

Text #2

Max Range: 6.1026. Min Range: -6.1026

Max Act: 5.9856. Min Act: -0.1700

Data Index: 3675105 (Open Web Text)

Max Activating Token Index: 976

Text #3

Max Range: 6.1026. Min Range: -6.1026

Max Act: 5.9503. Min Act: -0.1696

Data Index: 7876418 (Open Web Text)

Max Activating Token Index: 658

Text #4

Max Range: 6.1026. Min Range: -6.1026

Max Act: 6.0075. Min Act: -0.1699

Data Index: 1618371 (Open Web Text)

Max Activating Token Index: 647

Text #5

Max Range: 6.1026. Min Range: -6.1026

Max Act: 6.0698. Min Act: -0.1696

Data Index: 3506575 (Open Web Text)

Max Activating Token Index: 651

Text #6

Max Range: 6.1026. Min Range: -6.1026

Max Act: 6.0269. Min Act: -0.1659

Data Index: 5174840 (Open Web Text)

Max Activating Token Index: 773

Text #7

Max Range: 6.1026. Min Range: -6.1026

Max Act: 5.8673. Min Act: -0.1670

Data Index: 7857098 (Open Web Text)

Max Activating Token Index: 99

Text #8

Max Range: 6.1026. Min Range: -6.1026

Max Act: 5.8422. Min Act: -0.1669

Data Index: 202674 (Open Web Text)

Max Activating Token Index: 540

Text #9

Max Range: 6.1026. Min Range: -6.1026

Max Act: 5.9738. Min Act: -0.1680

Data Index: 1212891 (Open Web Text)

Max Activating Token Index: 489

Text #10

Max Range: 6.1026. Min Range: -6.1026

Max Act: 5.9256. Min Act: -0.1700

Data Index: 53198 (Open Web Text)

Max Activating Token Index: 565

Text #11

Max Range: 6.1026. Min Range: -6.1026

Max Act: 5.9284. Min Act: -0.1696

Data Index: 4604583 (Open Web Text)

Max Activating Token Index: 668

Text #12

Max Range: 6.1026. Min Range: -6.1026

Max Act: 5.8640. Min Act: -0.1698

Data Index: 1830855 (Open Web Text)

Max Activating Token Index: 578

Text #13

Max Range: 6.1026. Min Range: -6.1026

Max Act: 5.8812. Min Act: -0.1699

Data Index: 7683933 (Open Web Text)

Max Activating Token Index: 608

Text #14

Max Range: 6.1026. Min Range: -6.1026

Max Act: 5.8289. Min Act: -0.1700

Data Index: 2950354 (Open Web Text)

Max Activating Token Index: 79

Text #15

Max Range: 6.1026. Min Range: -6.1026

Max Act: 5.8506. Min Act: -0.1698

Data Index: 136929 (Open Web Text)

Max Activating Token Index: 1009

Text #16

Max Range: 6.1026. Min Range: -6.1026

Max Act: 5.8867. Min Act: -0.1696

Data Index: 7044023 (Open Web Text)

Max Activating Token Index: 551

Text #17

Max Range: 6.1026. Min Range: -6.1026

Max Act: 5.7767. Min Act: -0.1699

Data Index: 7594530 (Open Web Text)

Max Activating Token Index: 750

Text #18

Max Range: 6.1026. Min Range: -6.1026

Max Act: 5.7977. Min Act: -0.1700

Data Index: 4904839 (Open Web Text)

Max Activating Token Index: 751

Text #19

Max Range: 6.1026. Min Range: -6.1026

Max Act: 5.8320. Min Act: -0.1700

Data Index: 4057900 (Open Web Text)

Max Activating Token Index: 11