Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 363 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.3724. Min Act: -0.1698

Data Index: 7277224 (Open Web Text)

Max Activating Token Index: 930

Text #1

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.3309. Min Act: -0.1696

Data Index: 5419304 (Open Web Text)

Max Activating Token Index: 998

Text #2

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.3270. Min Act: -0.1670

Data Index: 3714886 (Open Web Text)

Max Activating Token Index: 934

Text #3

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.3206. Min Act: -0.1699

Data Index: 425915 (Open Web Text)

Max Activating Token Index: 895

Text #4

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.2817. Min Act: -0.1691

Data Index: 4855485 (Open Web Text)

Max Activating Token Index: 222

Text #5

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.2768. Min Act: -0.1700

Data Index: 7277223 (Open Web Text)

Max Activating Token Index: 854

Text #6

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.2690. Min Act: -0.1698

Data Index: 5419301 (Open Web Text)

Max Activating Token Index: 347

Text #7

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.2655. Min Act: -0.1698

Data Index: 1468930 (Open Web Text)

Max Activating Token Index: 765

Text #8

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.2644. Min Act: -0.1686

Data Index: 1468927 (Open Web Text)

Max Activating Token Index: 819

Text #9

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.2438. Min Act: -0.1698

Data Index: 1468929 (Open Web Text)

Max Activating Token Index: 790

Text #10

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.2517. Min Act: -0.1694

Data Index: 425914 (Open Web Text)

Max Activating Token Index: 1003

Text #11

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.2468. Min Act: -0.1700

Data Index: 5879769 (Open Web Text)

Max Activating Token Index: 995

Text #12

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.2246. Min Act: -0.1695

Data Index: 7277222 (Open Web Text)

Max Activating Token Index: 781

Text #13

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.2299. Min Act: -0.1700

Data Index: 3464796 (Open Web Text)

Max Activating Token Index: 205

Text #14

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.2330. Min Act: -0.1699

Data Index: 7964928 (Open Web Text)

Max Activating Token Index: 893

Text #15

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.2309. Min Act: -0.1691

Data Index: 5790730 (Open Web Text)

Max Activating Token Index: 397

Text #16

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.2063. Min Act: -0.1647

Data Index: 169063 (Open Web Text)

Max Activating Token Index: 137

Text #17

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.2050. Min Act: -0.1699

Data Index: 425913 (Open Web Text)

Max Activating Token Index: 1016

Text #18

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.2062. Min Act: -0.1691

Data Index: 3192432 (Open Web Text)

Max Activating Token Index: 271

Text #19

Max Range: 2.3724. Min Range: -2.3724

Max Act: 2.2033. Min Act: -0.1700

Data Index: 4148120 (Open Web Text)

Max Activating Token Index: 556