Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1319 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.5663. Min Act: -0.1700

Data Index: 4312307 (Open Web Text)

Max Activating Token Index: 735

Text #1

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4661. Min Act: -0.1700

Data Index: 6259441 (Open Web Text)

Max Activating Token Index: 532

Text #2

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4448. Min Act: -0.1700

Data Index: 6001648 (Open Web Text)

Max Activating Token Index: 200

Text #3

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4682. Min Act: -0.1700

Data Index: 5781606 (Open Web Text)

Max Activating Token Index: 442

Text #4

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4635. Min Act: -0.1700

Data Index: 6191603 (Open Web Text)

Max Activating Token Index: 856

Text #5

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4483. Min Act: -0.1700

Data Index: 3062920 (Open Web Text)

Max Activating Token Index: 410

Text #6

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4243. Min Act: -0.1700

Data Index: 1886793 (Open Web Text)

Max Activating Token Index: 829

Text #7

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4291. Min Act: -0.1700

Data Index: 7270886 (Open Web Text)

Max Activating Token Index: 902

Text #8

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4470. Min Act: -0.1700

Data Index: 1568651 (Open Web Text)

Max Activating Token Index: 441

Text #9

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4157. Min Act: -0.1700

Data Index: 6204501 (Open Web Text)

Max Activating Token Index: 980

Text #10

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4014. Min Act: -0.1700

Data Index: 6047654 (Open Web Text)

Max Activating Token Index: 416

Text #11

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4041. Min Act: -0.1700

Data Index: 6880283 (Open Web Text)

Max Activating Token Index: 519

Text #12

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4240. Min Act: -0.1700

Data Index: 7497760 (Open Web Text)

Max Activating Token Index: 510

Text #13

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4229. Min Act: -0.1700

Data Index: 2788133 (Open Web Text)

Max Activating Token Index: 917

Text #14

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4221. Min Act: -0.1700

Data Index: 684488 (Open Web Text)

Max Activating Token Index: 479

Text #15

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4195. Min Act: -0.1700

Data Index: 7689601 (Open Web Text)

Max Activating Token Index: 581

Text #16

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.3954. Min Act: -0.1700

Data Index: 4749407 (Open Web Text)

Max Activating Token Index: 529

Text #17

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4089. Min Act: -0.1700

Data Index: 3549520 (Open Web Text)

Max Activating Token Index: 512

Text #18

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4106. Min Act: -0.1700

Data Index: 1175301 (Open Web Text)

Max Activating Token Index: 451

Text #19

Max Range: 3.5663. Min Range: -3.5663

Max Act: 3.4058. Min Act: -0.1700

Data Index: 5779762 (Open Web Text)

Max Activating Token Index: 937