Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2549 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.9217. Min Act: -0.1700

Data Index: 1366444 (Open Web Text)

Max Activating Token Index: 890

Text #1

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.9158. Min Act: -0.1698

Data Index: 7419071 (Open Web Text)

Max Activating Token Index: 450

Text #2

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.9248. Min Act: -0.1700

Data Index: 7419067 (Open Web Text)

Max Activating Token Index: 1004

Text #3

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.8917. Min Act: -0.1700

Data Index: 6000887 (Open Web Text)

Max Activating Token Index: 308

Text #4

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.9027. Min Act: -0.1700

Data Index: 5035029 (Open Web Text)

Max Activating Token Index: 1023

Text #5

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.9095. Min Act: -0.1697

Data Index: 471950 (Open Web Text)

Max Activating Token Index: 499

Text #6

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.8999. Min Act: -0.1699

Data Index: 3713073 (Open Web Text)

Max Activating Token Index: 825

Text #7

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.8976. Min Act: -0.1700

Data Index: 1865044 (Open Web Text)

Max Activating Token Index: 886

Text #8

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.9106. Min Act: -0.1700

Data Index: 3713074 (Open Web Text)

Max Activating Token Index: 363

Text #9

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.8801. Min Act: -0.1700

Data Index: 6095815 (Open Web Text)

Max Activating Token Index: 694

Text #10

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.8846. Min Act: -0.1700

Data Index: 3979149 (Open Web Text)

Max Activating Token Index: 913

Text #11

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.8671. Min Act: -0.1700

Data Index: 6216912 (Open Web Text)

Max Activating Token Index: 523

Text #12

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.8685. Min Act: -0.1700

Data Index: 1366448 (Open Web Text)

Max Activating Token Index: 481

Text #13

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.8681. Min Act: -0.1700

Data Index: 1917688 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.8444. Min Act: -0.1700

Data Index: 471949 (Open Web Text)

Max Activating Token Index: 817

Text #15

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.8438. Min Act: -0.1700

Data Index: 2687396 (Open Web Text)

Max Activating Token Index: 424

Text #16

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.8437. Min Act: -0.1700

Data Index: 471955 (Open Web Text)

Max Activating Token Index: 281

Text #17

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.8505. Min Act: -0.1700

Data Index: 3692783 (Open Web Text)

Max Activating Token Index: 405

Text #18

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.8272. Min Act: -0.1700

Data Index: 7697291 (Open Web Text)

Max Activating Token Index: 524

Text #19

Max Range: 1.9248. Min Range: -1.9248

Max Act: 1.8432. Min Act: -0.1700

Data Index: 8572844 (Open Web Text)

Max Activating Token Index: 181