Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1613 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.6236. Min Act: -0.1700

Data Index: 3268578 (Open Web Text)

Max Activating Token Index: 623

Text #1

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.5450. Min Act: -0.1700

Data Index: 2594176 (Open Web Text)

Max Activating Token Index: 427

Text #2

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.5333. Min Act: -0.1700

Data Index: 5834694 (Open Web Text)

Max Activating Token Index: 1019

Text #3

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.5137. Min Act: -0.1700

Data Index: 5290575 (Open Web Text)

Max Activating Token Index: 9

Text #4

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.5110. Min Act: -0.1700

Data Index: 5118031 (Open Web Text)

Max Activating Token Index: 845

Text #5

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.5266. Min Act: -0.1700

Data Index: 669521 (Open Web Text)

Max Activating Token Index: 494

Text #6

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.4996. Min Act: -0.1700

Data Index: 7972508 (Open Web Text)

Max Activating Token Index: 933

Text #7

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.5173. Min Act: -0.1700

Data Index: 1381420 (Open Web Text)

Max Activating Token Index: 949

Text #8

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.4774. Min Act: -0.1700

Data Index: 2987495 (Open Web Text)

Max Activating Token Index: 455

Text #9

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.4411. Min Act: -0.1700

Data Index: 4864139 (Open Web Text)

Max Activating Token Index: 377

Text #10

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.4310. Min Act: -0.1700

Data Index: 2888979 (Open Web Text)

Max Activating Token Index: 499

Text #11

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.4690. Min Act: -0.1700

Data Index: 1267270 (Open Web Text)

Max Activating Token Index: 633

Text #12

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.5019. Min Act: -0.1700

Data Index: 1197143 (Open Web Text)

Max Activating Token Index: 975

Text #13

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.4996. Min Act: -0.1700

Data Index: 3463902 (Open Web Text)

Max Activating Token Index: 600

Text #14

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.4582. Min Act: -0.1700

Data Index: 2389372 (Open Web Text)

Max Activating Token Index: 474

Text #15

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.4358. Min Act: -0.1700

Data Index: 1627801 (Open Web Text)

Max Activating Token Index: 1000

Text #16

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.4200. Min Act: -0.1700

Data Index: 1698808 (Open Web Text)

Max Activating Token Index: 456

Text #17

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.4216. Min Act: -0.1700

Data Index: 5191054 (Open Web Text)

Max Activating Token Index: 160

Text #18

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.4194. Min Act: -0.1700

Data Index: 252551 (Open Web Text)

Max Activating Token Index: 551

Text #19

Max Range: 5.6236. Min Range: -5.6236

Max Act: 5.4703. Min Act: -0.1700

Data Index: 1113281 (Open Web Text)

Max Activating Token Index: 997