Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1988 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.7164. Min Act: -0.1700

Data Index: 7854903 (Open Web Text)

Max Activating Token Index: 18

Text #1

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.6697. Min Act: -0.1700

Data Index: 8461372 (Open Web Text)

Max Activating Token Index: 943

Text #2

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.6961. Min Act: -0.1700

Data Index: 2523192 (Open Web Text)

Max Activating Token Index: 54

Text #3

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.6400. Min Act: -0.1700

Data Index: 1761807 (Open Web Text)

Max Activating Token Index: 482

Text #4

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.5763. Min Act: -0.1700

Data Index: 7461023 (Open Web Text)

Max Activating Token Index: 424

Text #5

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.5821. Min Act: -0.1700

Data Index: 3667122 (Open Web Text)

Max Activating Token Index: 154

Text #6

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.6084. Min Act: -0.1700

Data Index: 8161512 (Open Web Text)

Max Activating Token Index: 603

Text #7

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.5779. Min Act: -0.1700

Data Index: 7213546 (Open Web Text)

Max Activating Token Index: 951

Text #8

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.5994. Min Act: -0.1700

Data Index: 5970138 (Open Web Text)

Max Activating Token Index: 442

Text #9

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.5784. Min Act: -0.1700

Data Index: 2843738 (Open Web Text)

Max Activating Token Index: 465

Text #10

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.5654. Min Act: -0.1700

Data Index: 383830 (Open Web Text)

Max Activating Token Index: 678

Text #11

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.5858. Min Act: -0.1700

Data Index: 2508078 (Open Web Text)

Max Activating Token Index: 684

Text #12

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.5417. Min Act: -0.1700

Data Index: 2323165 (Open Web Text)

Max Activating Token Index: 481

Text #13

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.5194. Min Act: -0.1700

Data Index: 8696683 (Open Web Text)

Max Activating Token Index: 138

Text #14

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.5277. Min Act: -0.1700

Data Index: 4487167 (Open Web Text)

Max Activating Token Index: 116

Text #15

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.4926. Min Act: -0.1700

Data Index: 6801332 (Open Web Text)

Max Activating Token Index: 625

Text #16

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.5154. Min Act: -0.1700

Data Index: 6834691 (Open Web Text)

Max Activating Token Index: 401

Text #17

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.5009. Min Act: -0.1700

Data Index: 2634324 (Open Web Text)

Max Activating Token Index: 659

Text #18

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.4973. Min Act: -0.1700

Data Index: 239377 (Open Web Text)

Max Activating Token Index: 797

Text #19

Max Range: 5.7164. Min Range: -5.7164

Max Act: 5.4903. Min Act: -0.1700

Data Index: 7258056 (Open Web Text)

Max Activating Token Index: 67