Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 263 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.6759. Min Act: -0.1700

Data Index: 4734416 (Open Web Text)

Max Activating Token Index: 401

Text #1

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.3705. Min Act: -0.1700

Data Index: 5994288 (Open Web Text)

Max Activating Token Index: 597

Text #2

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.3311. Min Act: -0.1700

Data Index: 5876933 (Open Web Text)

Max Activating Token Index: 231

Text #3

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.2918. Min Act: -0.1700

Data Index: 3691887 (Open Web Text)

Max Activating Token Index: 243

Text #4

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.3080. Min Act: -0.1700

Data Index: 6845856 (Open Web Text)

Max Activating Token Index: 598

Text #5

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.3120. Min Act: -0.1700

Data Index: 8059234 (Open Web Text)

Max Activating Token Index: 100

Text #6

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.2945. Min Act: -0.1700

Data Index: 7106329 (Open Web Text)

Max Activating Token Index: 947

Text #7

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.3029. Min Act: -0.1700

Data Index: 7582986 (Open Web Text)

Max Activating Token Index: 956

Text #8

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.3069. Min Act: -0.1700

Data Index: 2570761 (Open Web Text)

Max Activating Token Index: 369

Text #9

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.2877. Min Act: -0.1700

Data Index: 6774661 (Open Web Text)

Max Activating Token Index: 187

Text #10

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.2773. Min Act: -0.1700

Data Index: 4636347 (Open Web Text)

Max Activating Token Index: 479

Text #11

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.2709. Min Act: -0.1700

Data Index: 1841249 (Open Web Text)

Max Activating Token Index: 139

Text #12

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.2728. Min Act: -0.1700

Data Index: 4905304 (Open Web Text)

Max Activating Token Index: 1009

Text #13

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.2662. Min Act: -0.1700

Data Index: 5994274 (Open Web Text)

Max Activating Token Index: 915

Text #14

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.2759. Min Act: -0.1700

Data Index: 5399779 (Open Web Text)

Max Activating Token Index: 813

Text #15

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.2446. Min Act: -0.1700

Data Index: 5994283 (Open Web Text)

Max Activating Token Index: 698

Text #16

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.2630. Min Act: -0.1700

Data Index: 3087943 (Open Web Text)

Max Activating Token Index: 25

Text #17

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.2692. Min Act: -0.1700

Data Index: 5876928 (Open Web Text)

Max Activating Token Index: 239

Text #18

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.2465. Min Act: -0.1700

Data Index: 4575004 (Open Web Text)

Max Activating Token Index: 801

Text #19

Max Range: 3.6759. Min Range: -3.6759

Max Act: 3.2631. Min Act: -0.1700

Data Index: 3226520 (Open Web Text)

Max Activating Token Index: 995