Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 201 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.3961. Min Range: -5.3961

Max Act: 5.3961. Min Act: -0.1700

Data Index: 1653510 (Open Web Text)

Max Activating Token Index: 181

Text #1

Max Range: 5.3961. Min Range: -5.3961

Max Act: 5.1959. Min Act: -0.1700

Data Index: 8442559 (Open Web Text)

Max Activating Token Index: 239

Text #2

Max Range: 5.3961. Min Range: -5.3961

Max Act: 5.0348. Min Act: -0.1700

Data Index: 2031627 (Open Web Text)

Max Activating Token Index: 764

Text #3

Max Range: 5.3961. Min Range: -5.3961

Max Act: 5.0048. Min Act: -0.1700

Data Index: 4268717 (Open Web Text)

Max Activating Token Index: 331

Text #4

Max Range: 5.3961. Min Range: -5.3961

Max Act: 5.0038. Min Act: -0.1700

Data Index: 5340757 (Open Web Text)

Max Activating Token Index: 255

Text #5

Max Range: 5.3961. Min Range: -5.3961

Max Act: 4.9926. Min Act: -0.1700

Data Index: 754401 (Open Web Text)

Max Activating Token Index: 542

Text #6

Max Range: 5.3961. Min Range: -5.3961

Max Act: 4.9712. Min Act: -0.1700

Data Index: 5288671 (Open Web Text)

Max Activating Token Index: 30

Text #7

Max Range: 5.3961. Min Range: -5.3961

Max Act: 4.8840. Min Act: -0.1700

Data Index: 5184861 (Open Web Text)

Max Activating Token Index: 68

Text #8

Max Range: 5.3961. Min Range: -5.3961

Max Act: 4.9126. Min Act: -0.1700

Data Index: 511265 (Open Web Text)

Max Activating Token Index: 82

Text #9

Max Range: 5.3961. Min Range: -5.3961

Max Act: 4.8415. Min Act: -0.1700

Data Index: 6132387 (Open Web Text)

Max Activating Token Index: 380

Text #10

Max Range: 5.3961. Min Range: -5.3961

Max Act: 4.9191. Min Act: -0.1700

Data Index: 5251517 (Open Web Text)

Max Activating Token Index: 90

Text #11

Max Range: 5.3961. Min Range: -5.3961

Max Act: 4.8913. Min Act: -0.1700

Data Index: 2691470 (Open Web Text)

Max Activating Token Index: 1000

Text #12

Max Range: 5.3961. Min Range: -5.3961

Max Act: 4.8874. Min Act: -0.1700

Data Index: 5340756 (Open Web Text)

Max Activating Token Index: 686

Text #13

Max Range: 5.3961. Min Range: -5.3961

Max Act: 4.8812. Min Act: -0.1700

Data Index: 3854999 (Open Web Text)

Max Activating Token Index: 399

Text #14

Max Range: 5.3961. Min Range: -5.3961

Max Act: 4.8644. Min Act: -0.1700

Data Index: 5836529 (Open Web Text)

Max Activating Token Index: 61

Text #15

Max Range: 5.3961. Min Range: -5.3961

Max Act: 4.8892. Min Act: -0.1700

Data Index: 1537386 (Open Web Text)

Max Activating Token Index: 954

Text #16

Max Range: 5.3961. Min Range: -5.3961

Max Act: 4.8540. Min Act: -0.1700

Data Index: 2707689 (Open Web Text)

Max Activating Token Index: 72

Text #17

Max Range: 5.3961. Min Range: -5.3961

Max Act: 4.8398. Min Act: -0.1700

Data Index: 1707740 (Open Web Text)

Max Activating Token Index: 666

Text #18

Max Range: 5.3961. Min Range: -5.3961

Max Act: 4.8606. Min Act: -0.1700

Data Index: 6254887 (Open Web Text)

Max Activating Token Index: 303

Text #19

Max Range: 5.3961. Min Range: -5.3961

Max Act: 4.8489. Min Act: -0.1700

Data Index: 4770477 (Open Web Text)

Max Activating Token Index: 349