Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 800 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.6152. Min Act: -0.1541

Data Index: 6707907 (Open Web Text)

Max Activating Token Index: 963

Text #1

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.5296. Min Act: -0.1680

Data Index: 2428670 (Open Web Text)

Max Activating Token Index: 666

Text #2

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4700. Min Act: -0.1228

Data Index: 4515519 (Open Web Text)

Max Activating Token Index: 729

Text #3

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4755. Min Act: -0.1098

Data Index: 4515536 (Open Web Text)

Max Activating Token Index: 1012

Text #4

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4719. Min Act: -0.1697

Data Index: 3179945 (Open Web Text)

Max Activating Token Index: 566

Text #5

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4596. Min Act: -0.1508

Data Index: 2977755 (Open Web Text)

Max Activating Token Index: 1014

Text #6

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4670. Min Act: -0.1502

Data Index: 8339776 (Open Web Text)

Max Activating Token Index: 622

Text #7

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4659. Min Act: -0.1587

Data Index: 2617612 (Open Web Text)

Max Activating Token Index: 290

Text #8

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4638. Min Act: -0.1655

Data Index: 1077483 (Open Web Text)

Max Activating Token Index: 620

Text #9

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4503. Min Act: -0.1638

Data Index: 1534139 (Open Web Text)

Max Activating Token Index: 694

Text #10

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4549. Min Act: -0.1336

Data Index: 5933398 (Open Web Text)

Max Activating Token Index: 612

Text #11

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4508. Min Act: -0.1439

Data Index: 674925 (Open Web Text)

Max Activating Token Index: 206

Text #12

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4563. Min Act: -0.1397

Data Index: 2711076 (Open Web Text)

Max Activating Token Index: 673

Text #13

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4549. Min Act: -0.1513

Data Index: 4550239 (Open Web Text)

Max Activating Token Index: 661

Text #14

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4546. Min Act: -0.1613

Data Index: 7535330 (Open Web Text)

Max Activating Token Index: 602

Text #15

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4553. Min Act: -0.1534

Data Index: 7502957 (Open Web Text)

Max Activating Token Index: 550

Text #16

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4536. Min Act: -0.0756

Data Index: 7502961 (Open Web Text)

Max Activating Token Index: 592

Text #17

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4538. Min Act: -0.1534

Data Index: 1230381 (Open Web Text)

Max Activating Token Index: 679

Text #18

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4580. Min Act: -0.0817

Data Index: 4515517 (Open Web Text)

Max Activating Token Index: 840

Text #19

Max Range: 1.6152. Min Range: -1.6152

Max Act: 1.4484. Min Act: -0.1383

Data Index: 461620 (Open Web Text)

Max Activating Token Index: 682