Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1223 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.5017. Min Act: -0.0992

Data Index: 954086 (C4 (Web Text))

Max Activating Token Index: 47

Text #1

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.3924. Min Act: -0.1347

Data Index: 1046677 (C4 (Web Text))

Max Activating Token Index: 986

Text #2

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.2847. Min Act: -0.1325

Data Index: 1086597 (C4 (Web Text))

Max Activating Token Index: 906

Text #3

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.2418. Min Act: -0.0899

Data Index: 1139112 (C4 (Web Text))

Max Activating Token Index: 603

Text #4

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.1981. Min Act: -0.1660

Data Index: 386023 (C4 (Web Text))

Max Activating Token Index: 868

Text #5

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.2164. Min Act: -0.1061

Data Index: 431788 (C4 (Web Text))

Max Activating Token Index: 309

Text #6

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.1790. Min Act: -0.1533

Data Index: 1155309 (C4 (Web Text))

Max Activating Token Index: 354

Text #7

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.1598. Min Act: -0.1700

Data Index: 1273103 (C4 (Web Text))

Max Activating Token Index: 607

Text #8

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.1302. Min Act: -0.1058

Data Index: 1210046 (C4 (Web Text))

Max Activating Token Index: 957

Text #9

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.1228. Min Act: -0.1700

Data Index: 468066 (C4 (Web Text))

Max Activating Token Index: 968

Text #10

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.1229. Min Act: 0.0358

Data Index: 45688 (C4 (Web Text))

Max Activating Token Index: 488

Text #11

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.1183. Min Act: -0.1229

Data Index: 809569 (C4 (Web Text))

Max Activating Token Index: 909

Text #12

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.1306. Min Act: -0.1193

Data Index: 746536 (C4 (Web Text))

Max Activating Token Index: 609

Text #13

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.1165. Min Act: -0.1233

Data Index: 877737 (C4 (Web Text))

Max Activating Token Index: 427

Text #14

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.0967. Min Act: -0.1405

Data Index: 946115 (C4 (Web Text))

Max Activating Token Index: 621

Text #15

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.0939. Min Act: -0.1530

Data Index: 208060 (C4 (Web Text))

Max Activating Token Index: 192

Text #16

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.0897. Min Act: -0.1262

Data Index: 691186 (C4 (Web Text))

Max Activating Token Index: 933

Text #17

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.0854. Min Act: -0.1700

Data Index: 441599 (C4 (Web Text))

Max Activating Token Index: 423

Text #18

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.0878. Min Act: -0.0280

Data Index: 1622914 (Python Code)

Max Activating Token Index: 928

Text #19

Max Range: 4.5017. Min Range: -4.5017

Max Act: 4.0789. Min Act: -0.1446

Data Index: 785926 (C4 (Web Text))

Max Activating Token Index: 460