Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2575 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.2061. Min Act: -0.1700

Data Index: 6635545 (Open Web Text)

Max Activating Token Index: 965

Text #1

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.1935. Min Act: -0.1700

Data Index: 5885600 (Open Web Text)

Max Activating Token Index: 510

Text #2

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.1995. Min Act: -0.1700

Data Index: 8738258 (Open Web Text)

Max Activating Token Index: 391

Text #3

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.1386. Min Act: -0.1700

Data Index: 3218068 (Open Web Text)

Max Activating Token Index: 789

Text #4

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.1262. Min Act: -0.1700

Data Index: 1017401 (Open Web Text)

Max Activating Token Index: 855

Text #5

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.1048. Min Act: -0.1700

Data Index: 5768179 (Open Web Text)

Max Activating Token Index: 830

Text #6

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.1362. Min Act: -0.1700

Data Index: 8738250 (Open Web Text)

Max Activating Token Index: 796

Text #7

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.1029. Min Act: -0.1700

Data Index: 5909968 (Open Web Text)

Max Activating Token Index: 399

Text #8

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.1099. Min Act: -0.1700

Data Index: 3186367 (Open Web Text)

Max Activating Token Index: 48

Text #9

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.1126. Min Act: -0.1700

Data Index: 353055 (Open Web Text)

Max Activating Token Index: 33

Text #10

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.0324. Min Act: -0.1700

Data Index: 6803228 (Open Web Text)

Max Activating Token Index: 505

Text #11

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.0901. Min Act: -0.1700

Data Index: 7248714 (Open Web Text)

Max Activating Token Index: 412

Text #12

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.0613. Min Act: -0.1700

Data Index: 6591408 (Open Web Text)

Max Activating Token Index: 63

Text #13

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.0632. Min Act: -0.1700

Data Index: 4993378 (Open Web Text)

Max Activating Token Index: 175

Text #14

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.0515. Min Act: -0.1700

Data Index: 6217926 (Open Web Text)

Max Activating Token Index: 912

Text #15

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.0886. Min Act: -0.1700

Data Index: 1913966 (Open Web Text)

Max Activating Token Index: 620

Text #16

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.0532. Min Act: -0.1700

Data Index: 8171994 (Open Web Text)

Max Activating Token Index: 507

Text #17

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.0479. Min Act: -0.1700

Data Index: 484851 (Open Web Text)

Max Activating Token Index: 367

Text #18

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.0475. Min Act: -0.1700

Data Index: 6578417 (Open Web Text)

Max Activating Token Index: 230

Text #19

Max Range: 3.2061. Min Range: -3.2061

Max Act: 3.0400. Min Act: -0.1700

Data Index: 4763537 (Open Web Text)

Max Activating Token Index: 605