Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 570 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.8038. Min Act: -0.1700

Data Index: 6686683 (Open Web Text)

Max Activating Token Index: 404

Text #1

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.7829. Min Act: -0.1700

Data Index: 8422267 (Open Web Text)

Max Activating Token Index: 535

Text #2

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.7457. Min Act: -0.1700

Data Index: 502563 (Open Web Text)

Max Activating Token Index: 85

Text #3

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.6554. Min Act: -0.1700

Data Index: 7618822 (Open Web Text)

Max Activating Token Index: 497

Text #4

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.6207. Min Act: -0.1700

Data Index: 1029885 (Open Web Text)

Max Activating Token Index: 363

Text #5

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.5732. Min Act: -0.1700

Data Index: 3936537 (Open Web Text)

Max Activating Token Index: 199

Text #6

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.5700. Min Act: -0.1700

Data Index: 2974352 (Open Web Text)

Max Activating Token Index: 59

Text #7

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.5368. Min Act: -0.1700

Data Index: 4436532 (Open Web Text)

Max Activating Token Index: 355

Text #8

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.5496. Min Act: -0.1700

Data Index: 2670783 (Open Web Text)

Max Activating Token Index: 703

Text #9

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.5152. Min Act: -0.1700

Data Index: 2550229 (Open Web Text)

Max Activating Token Index: 484

Text #10

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.4794. Min Act: -0.1700

Data Index: 2962206 (Open Web Text)

Max Activating Token Index: 560

Text #11

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.4806. Min Act: -0.1700

Data Index: 4954300 (Open Web Text)

Max Activating Token Index: 553

Text #12

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.4951. Min Act: -0.1700

Data Index: 2424194 (Open Web Text)

Max Activating Token Index: 1022

Text #13

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.4844. Min Act: -0.1700

Data Index: 8528938 (Open Web Text)

Max Activating Token Index: 607

Text #14

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.5271. Min Act: -0.1700

Data Index: 2012736 (Open Web Text)

Max Activating Token Index: 923

Text #15

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.4897. Min Act: -0.1700

Data Index: 8731876 (Open Web Text)

Max Activating Token Index: 592

Text #16

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.4804. Min Act: -0.1700

Data Index: 8274093 (Open Web Text)

Max Activating Token Index: 321

Text #17

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.5011. Min Act: -0.1700

Data Index: 7647524 (Open Web Text)

Max Activating Token Index: 704

Text #18

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.4345. Min Act: -0.1700

Data Index: 3894172 (Open Web Text)

Max Activating Token Index: 142

Text #19

Max Range: 3.8038. Min Range: -3.8038

Max Act: 3.4733. Min Act: -0.1700

Data Index: 1179126 (Open Web Text)

Max Activating Token Index: 518