Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1797 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.7178. Min Act: -0.1700

Data Index: 7985962 (Open Web Text)

Max Activating Token Index: 365

Text #1

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.6790. Min Act: -0.1699

Data Index: 7985968 (Open Web Text)

Max Activating Token Index: 280

Text #2

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.6659. Min Act: -0.1700

Data Index: 7985958 (Open Web Text)

Max Activating Token Index: 793

Text #3

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.6456. Min Act: -0.1700

Data Index: 4685762 (Open Web Text)

Max Activating Token Index: 340

Text #4

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.6080. Min Act: -0.1699

Data Index: 7985961 (Open Web Text)

Max Activating Token Index: 850

Text #5

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.5873. Min Act: -0.1700

Data Index: 4812234 (Open Web Text)

Max Activating Token Index: 592

Text #6

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.5899. Min Act: -0.1695

Data Index: 7985965 (Open Web Text)

Max Activating Token Index: 827

Text #7

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.5796. Min Act: -0.1700

Data Index: 8755337 (Open Web Text)

Max Activating Token Index: 130

Text #8

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.5888. Min Act: -0.1700

Data Index: 3060900 (Open Web Text)

Max Activating Token Index: 561

Text #9

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.5715. Min Act: -0.1700

Data Index: 8753227 (Open Web Text)

Max Activating Token Index: 305

Text #10

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.5705. Min Act: -0.1700

Data Index: 7377075 (Open Web Text)

Max Activating Token Index: 359

Text #11

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.5625. Min Act: -0.1700

Data Index: 3070747 (Open Web Text)

Max Activating Token Index: 700

Text #12

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.5684. Min Act: -0.1698

Data Index: 4051433 (Open Web Text)

Max Activating Token Index: 273

Text #13

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.5740. Min Act: -0.1699

Data Index: 1425207 (Open Web Text)

Max Activating Token Index: 574

Text #14

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.5651. Min Act: -0.1695

Data Index: 4514439 (Open Web Text)

Max Activating Token Index: 306

Text #15

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.5703. Min Act: -0.1700

Data Index: 1425209 (Open Web Text)

Max Activating Token Index: 488

Text #16

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.5630. Min Act: -0.1699

Data Index: 5924270 (Open Web Text)

Max Activating Token Index: 310

Text #17

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.5735. Min Act: -0.1700

Data Index: 3060899 (Open Web Text)

Max Activating Token Index: 555

Text #18

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.5716. Min Act: -0.1700

Data Index: 178657 (Open Web Text)

Max Activating Token Index: 530

Text #19

Max Range: 1.7178. Min Range: -1.7178

Max Act: 1.5659. Min Act: -0.1700

Data Index: 6443084 (Open Web Text)

Max Activating Token Index: 650