Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 313 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.5436. Min Act: -0.1700

Data Index: 7646145 (Open Web Text)

Max Activating Token Index: 967

Text #1

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.5308. Min Act: -0.1700

Data Index: 4178844 (Open Web Text)

Max Activating Token Index: 73

Text #2

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.4895. Min Act: -0.1700

Data Index: 8229704 (Open Web Text)

Max Activating Token Index: 370

Text #3

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.4839. Min Act: -0.1700

Data Index: 4267688 (Open Web Text)

Max Activating Token Index: 630

Text #4

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.4854. Min Act: -0.1700

Data Index: 5440089 (Open Web Text)

Max Activating Token Index: 362

Text #5

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.4480. Min Act: -0.1700

Data Index: 7552323 (Open Web Text)

Max Activating Token Index: 49

Text #6

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.4567. Min Act: -0.1700

Data Index: 3463023 (Open Web Text)

Max Activating Token Index: 983

Text #7

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.4589. Min Act: -0.1700

Data Index: 3355971 (Open Web Text)

Max Activating Token Index: 226

Text #8

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.3825. Min Act: -0.1700

Data Index: 4628869 (Open Web Text)

Max Activating Token Index: 757

Text #9

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.4420. Min Act: -0.1700

Data Index: 7387327 (Open Web Text)

Max Activating Token Index: 76

Text #10

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.4481. Min Act: -0.1700

Data Index: 6613872 (Open Web Text)

Max Activating Token Index: 800

Text #11

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.4148. Min Act: -0.1700

Data Index: 2304978 (Open Web Text)

Max Activating Token Index: 15

Text #12

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.3694. Min Act: -0.1700

Data Index: 7294455 (Open Web Text)

Max Activating Token Index: 327

Text #13

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.3930. Min Act: -0.1700

Data Index: 8438901 (Open Web Text)

Max Activating Token Index: 765

Text #14

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.3824. Min Act: -0.1700

Data Index: 78256 (Open Web Text)

Max Activating Token Index: 657

Text #15

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.4017. Min Act: -0.1700

Data Index: 1480979 (Open Web Text)

Max Activating Token Index: 73

Text #16

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.3673. Min Act: -0.1700

Data Index: 7246886 (Open Web Text)

Max Activating Token Index: 545

Text #17

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.3577. Min Act: -0.1700

Data Index: 1298981 (Open Web Text)

Max Activating Token Index: 308

Text #18

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.3508. Min Act: -0.1700

Data Index: 613458 (Open Web Text)

Max Activating Token Index: 386

Text #19

Max Range: 4.5436. Min Range: -4.5436

Max Act: 4.3650. Min Act: -0.1700

Data Index: 2328654 (Open Web Text)

Max Activating Token Index: 255